Reporting standards in medical journals

Reporting standards in medical journals

Reporting standards in medical journals

On this page

P-values & CIs - The Dynamic Duo

  • P-value: Probability of obtaining observed results (or more extreme) if the null hypothesis is true.
    • Significance threshold: $p < 0.05$.
    • Indicates strength of evidence against the null hypothesis.
  • Confidence Interval (CI): Range of plausible values for a population parameter (e.g., mean or odds ratio).
    • A 95% CI implies a 95% probability that this range contains the true value.
    • If the CI for a difference doesn't include 0 (or 1 for a ratio), the result is statistically significant at the corresponding alpha level.

⭐ A confidence interval is superior to a p-value because it conveys both statistical significance and the precision of the effect estimate.

Reporting Pitfalls - Common Journal Gaffes

  • P-hacking (Selective Reporting): Reporting only favorable data or analyses that yield significant p-values, creating a biased view of the evidence.
  • Misinterpreting Non-Significance: Incorrectly concluding "no effect" or "no association" when a p-value is > 0.05. It only means the observed data are not sufficient to reject the null hypothesis.
  • Confusing Statistical vs. Clinical Significance: A small p-value (e.g., p < 0.001) doesn't guarantee a large or clinically meaningful effect. Large sample sizes can make trivial effects statistically significant.
  • Omitting Confidence Intervals: Reporting only a p-value without the CI hides the precision and magnitude of the effect estimate. A wide CI indicates high uncertainty.

⭐ Absence of evidence is not evidence of absence. A non-significant p-value does not prove the null hypothesis is true.

The Rulebook - Journal Reporting Standards

  • P-values:

    • Report exact values (e.g., p=0.02), not just thresholds (p < 0.05).
    • State the pre-specified significance level (α), usually 0.05.
    • Avoid misinterpreting the p-value as the probability that the null hypothesis is true.
  • Confidence Intervals (CIs):

    • Report CIs for all primary effect estimates (e.g., Relative Risk, Odds Ratio).
    • The 95% CI provides a range of plausible values for the true effect and indicates the precision of the estimate.
  • Reporting Guidelines:

    • Adhere to CONSORT (CONsolidated Standards of Reporting Trials) for RCTs to ensure transparency and completeness. Adherence is mandated by most major journals.

⭐ If the 95% CI for a ratio (e.g., OR, RR) does not contain the null value of 1.0, the result is statistically significant (p < 0.05).

Good vs. Bad - A Reporting Showdown

Good Reporting (Informative)Bad Reporting (Misleading)
Report exact p-value: e.g., $p=0.03## Good vs. Bad - A Reporting Showdown

| Imprecise statements: $p < 0.05## Good vs. Bad - A Reporting Showdown

| | Provide effect size & 95% CI: e.g., RR 1.5 (95% CI 1.1-2.1) | Isolate p-values: No context of effect size or CI | | Interpret CI: Focus on range of possible effects | Binary thinking: Equating non-significance with "no effect" |

⭐ If the 95% CI for a mean difference contains 0 (or for an odds/risk ratio contains 1), the result is not statistically significant ($p > 0.05$).

  • Report exact p-values (e.g., p=0.02) instead of just thresholds (e.g., p<0.05).
  • Confidence intervals (CIs) are superior to p-values, showing effect size and precision.
  • A statistically significant result has a 95% CI that excludes the null value.
  • The null value is 0 for a difference (e.g., mean difference).
  • The null value is 1 for a ratio (e.g., odds ratio, relative risk).
  • CIs provide the range of plausible values for the true effect.

Practice Questions: Reporting standards in medical journals

Test your understanding with these related questions

A study is funded by the tobacco industry to examine the association between smoking and lung cancer. They design a study with a prospective cohort of 1,000 smokers between the ages of 20-30. The length of the study is five years. After the study period ends, they conclude that there is no relationship between smoking and lung cancer. Which of the following study features is the most likely reason for the failure of the study to note an association between tobacco use and cancer?

1 of 5

Flashcards: Reporting standards in medical journals

1/7

_____ studies are useful for calculating relative risk (RR)

TAP TO REVEAL ANSWER

_____ studies are useful for calculating relative risk (RR)

Cohort

browseSpaceflip

Enjoying this lesson?

Get full access to all lessons, practice questions, and more.

Start Your Free Trial