You are conducting a study comparing the efficacy of two different statin medications. Two groups are placed on different statin medications, statin A and statin B. Baseline LDL levels are drawn for each group and are subsequently measured every 3 months for 1 year. Average baseline LDL levels for each group were identical. The group receiving statin A exhibited an 11 mg/dL greater reduction in LDL in comparison to the statin B group. Your statistical analysis reports a p-value of 0.052. Which of the following best describes the meaning of this p-value?

There is a 5.2% chance of observing a difference in reduction of LDL of 11 mg/dL or greater even if the two medications have identical effects

There is a 95% chance that the difference in reduction of LDL observed reflects a real difference between the two groups

Though A is more effective than B, there is a 5% chance the difference in reduction of LDL between the two groups is due to chance

If 100 permutations of this experiment were conducted, 5 of them would show similar results to those described above

This is a statistically significant result

A randomized control double-blind study is conducted on the efficacy of 2 sulfonylureas. The study concluded that medication 1 was more efficacious in lowering fasting blood glucose than medication 2 (p ≤ 0.05; 95% CI: 14 [10-21]). Which of the following is true regarding a 95% confidence interval (CI)?

If the same study were repeated multiple times, approximately 95% of the calculated confidence intervals would contain the true population parameter.

The 95% confidence interval is the probability chosen by the researcher to be the threshold of statistical significance.

When a 95% CI for the estimated difference between groups contains the value ‘0’, the results are significant.

It represents the probability that chance would not produce the difference shown, 95% of the time.

The study is adequately powered at the 95% confidence interval.

You submit a paper to a prestigious journal about the effects of coffee consumption on mesothelioma risk. The first reviewer lauds your clinical and scientific acumen, but expresses concern that your study does not have adequate statistical power. Statistical power refers to which of the following?

The probability of detecting an association when an association does exist.

The probability of detecting an association when no association exists.

The probability of not detecting an association when an association does exist.

The square root of the variance.

Two research groups independently study the same genetic variant's association with diabetes. Study A (n=5,000) reports OR=1.25, 95% CI: 1.05-1.48, p=0.01. Study B (n=50,000) reports OR=1.08, 95% CI: 1.02-1.14, p=0.006. Both studies are methodologically sound. Synthesize these findings to determine the most likely true effect and evaluate implications for clinical and research interpretation.

The true effect is likely modest (closer to Study B's estimate); Study A likely overestimated due to smaller sample size, but both show statistical significance with clinically marginal effects

Study B is definitive because of its larger sample size and should replace Study A's findings

The study with the lower p-value (Study B) is automatically more reliable

The studies are contradictory and no conclusions can be drawn

Study A is correct because it was published first

A prestigious journal publishes a trial showing a new cancer drug extends survival by 2 months (p=0.001, 95% CI: 1.5-2.5 months). The drug costs $150,000 per patient and causes Grade 3-4 toxicity in 60% of patients. Three prior unpublished trials showed non-significant results (all p>0.20). Synthesize these findings to evaluate the evidence base.

This pattern suggests publication bias; the significant result may be a false positive among multiple trials, and the modest benefit must be weighed against substantial toxicity and cost

The confidence interval proves the drug should be standard of care

P-values below 0.01 override concerns about prior negative studies

The published study's highly significant p-value validates the drug's efficacy

The three unpublished trials are irrelevant to evaluating the published study

A pharmaceutical company conducts 20 different analyses on their trial data, testing for effects on various secondary outcomes. One analysis shows a significant benefit (p=0.03) on hospital readmission rates. The primary outcome (mortality) showed p=0.12. The company seeks FDA approval based on the readmission data. Evaluate the validity and implications of this approach.

This represents multiple testing without correction, inflating Type I error; the significant result may be due to chance and selective reporting

Secondary outcomes are more important than primary outcomes when significant

The p=0.03 result is valid and supports approval regardless of the primary outcome

Any p<0.05 in a clinical trial justifies approval

The mortality p-value of 0.12 is close enough to significance to support both findings

Clinical vs statistical significance for USMLE Step 1: High-Yield Biostatistics Lessons

P-Values - Numbers Have Significance

P-value: The probability of observing data as extreme (or more extreme) than the current results, assuming the null hypothesis (H₀) is true.
- Null Hypothesis (H₀): Assumes no difference or effect (e.g., a new drug is no better than a placebo).
- Alternative Hypothesis (H₁): Assumes a difference or effect exists.
Significance Threshold (α): Typically set at 0.05.
- If $p < 0.05$, the results are statistically significant.
- This allows for the rejection of the null hypothesis (H₀).

Normal distribution with rejection regions

⭐ Statistical significance is not the same as clinical significance. A very large study might find a statistically significant result that is too small to be clinically meaningful (e.g., a drug that lowers blood pressure by only 1 mmHg).

Confidence Intervals - Guessing with Precision

Confidence Interval (CI): An estimated range of values that is likely to contain the true population parameter. It quantifies the uncertainty around an estimate.
A 95% CI implies that if a study were repeated many times, 95% of the calculated CIs would contain the true population value.
Relationship to p-value:
- If the 95% CI for an effect size does NOT cross the null value, the result is statistically significant (p < 0.05).
- Null values: 1 for ratios (Odds Ratio, Relative Risk), 0 for differences (means).
Precision & CI Width:
- Narrow CI: ↑ precision (less random error).
- Wide CI: ↓ precision (more random error).

Average TV hours per week with 95% confidence interval

⭐ If the CIs for two different groups do not overlap, their difference is statistically significant. However, if they do overlap, the difference may or may not be statistically significant-you must calculate the CI for the difference itself.

Clinical vs. Statistical - Real World vs. Research

Statistical Significance: Is the observed effect likely due to chance? Governed by the $p$-value.
Clinical Significance: Is the effect large enough to be meaningful for patients and change clinical practice? Governed by effect size.

Hypothesis Testing with Distribution and Rejection Regions

Feature	Statistical Significance	Clinical Significance
Question Answered	Is the effect real (not by chance)?	Does the effect matter in practice?
Key Metric	$p$-value	Effect size (e.g., NNT, odds ratio)
Key Influencer	Sample size	Magnitude of benefit, patient values
Interpretation	If p < 0.05, result is unlikely random	A small effect may be statistically significant but clinically irrelevant

Statistical significance (p < 0.05) just means a finding is unlikely to be due to chance; it does not automatically imply clinical importance.

Clinical significance is the practical relevance of a treatment effect-whether it makes a real difference to patients.

Very large sample sizes can make tiny, clinically irrelevant effects statistically significant.

Conversely, a small study may be underpowered and fail to find statistical significance for a clinically important effect.

Evaluate the effect size (e.g., relative risk) and the confidence interval to gauge clinical relevance.

Continue reading on Oncourse

CONTINUE READING — FREE

or get the app

App Store|Google Play