A randomized double-blind controlled trial is conducted on the efficacy of 2 different ACE-inhibitors. The null hypothesis is that both drugs will be equivalent in their blood-pressure-lowering abilities. The study concluded, however, that Medication 1 was more efficacious in lowering blood pressure than medication 2 as determined by a p-value < 0.01 (with significance defined as p ≤ 0.05). Which of the following statements is correct?

We can reject the null hypothesis.

We can accept the null hypothesis.

This trial did not reach statistical significance.

There is a 0.1% chance that medication 2 is superior.

There is a 10% chance that medication 1 is superior.

A study is performed to determine the prevalence of a particular rare fungal pneumonia. A sample population of 100 subjects is monitored for 4 months. Every month, the entire population is screened and the number of new cases is recorded for the group. The data from the study are given in the table below: Time point New cases of fungal pneumonia t = 0 months 10 t = 1 months 4 t = 2 months 2 t = 3 months 5 t = 4 months 4 Which of the following is correct regarding the prevalence of this rare fungal pneumonia in this sample population?

The prevalence at the conclusion of the study is 25%.

The prevalence at time point 2 months is 2%.

The prevalence at time point 3 months is 11%.

The prevalence at the conclusion of the study is 15%.

The prevalence and the incidence at time point 2 months are equal.

You are conducting a study comparing the efficacy of two different statin medications. Two groups are placed on different statin medications, statin A and statin B. Baseline LDL levels are drawn for each group and are subsequently measured every 3 months for 1 year. Average baseline LDL levels for each group were identical. The group receiving statin A exhibited an 11 mg/dL greater reduction in LDL in comparison to the statin B group. Your statistical analysis reports a p-value of 0.052. Which of the following best describes the meaning of this p-value?

There is a 5.2% chance of observing a difference in reduction of LDL of 11 mg/dL or greater even if the two medications have identical effects

There is a 95% chance that the difference in reduction of LDL observed reflects a real difference between the two groups

Though A is more effective than B, there is a 5% chance the difference in reduction of LDL between the two groups is due to chance

If 100 permutations of this experiment were conducted, 5 of them would show similar results to those described above

This is a statistically significant result

A researcher is conducting a study to compare fracture risk in male patients above the age of 65 who received annual DEXA screening to peers who did not receive screening. He conducts a randomized controlled trial in 900 patients, with half of participants assigned to each experimental group. The researcher ultimately finds similar rates of fractures in the two groups. He then notices that he had forgotten to include 400 patients in his analysis. Including the additional participants in his analysis would most likely affect the study's results in which of the following ways?

Increased probability of rejecting the null hypothesis when it is truly false

Wider confidence intervals of results

Increased probability of committing a type II error

Decreased significance level of results

Increased external validity of results

Effect size and clinical significance for USMLE Step 2 CK: High-Yield Behavioral Science Lessons

Effect Size - Beyond P-Value Power

Effect size measures the magnitude of an intervention's effect or the strength of a relationship, unlike p-value, which only indicates if an effect is statistically significant (not due to chance).
It answers, "How much does it matter?" not just "Is there an effect?". It is a key indicator of clinical significance.

Common Measures:
- Cohen's d: For differences between two means. Benchmarks: Small (0.2), Medium (0.5), Large (0.8).
- Odds Ratio (OR) / Relative Risk (RR): For proportions in categorical data.
- Correlation Coefficient (r): For strength of association between two variables.

Cohen's d and Overlap for Small, Medium, and Large Effects

⭐ A very large sample size can make a clinically insignificant effect statistically significant (e.g., p < 0.05), but the effect size will remain small, revealing its limited practical importance.

Quantifying Impact - The Effect Size Zoo

Effect Size: Measures the magnitude of an intervention's effect or a relationship's strength. It is independent of sample size and crucial for assessing clinical significance, unlike the p-value.
Common Metrics for Group Differences:
- Cohen's d: Standardized difference between two means.
  - Interpretation: Small (0.2), Medium (0.5), Large (0.8).
Common Metrics for Association (Ratios):
- Odds Ratio (OR): Used in case-control studies. Odds of an event in one group vs. another.
- Relative Risk (RR): Used in cohort studies. Probability of an event in an exposed group vs. an unexposed group.
- Hazard Ratio (HR): Used in survival analysis. Instantaneous risk over time.
Interpreting Ratios (OR, RR, HR):
- 1: ↑ risk/odds.
- < 1: ↓ risk/odds (protective).
- = 1: No difference.

⭐ For rare diseases (low prevalence), the Odds Ratio (OR) from a case-control study closely approximates the Relative Risk (RR).

Clinical Significance - The 'So What?' Test

Statistical significance (p < 0.05) indicates if an effect is likely due to chance, but not its magnitude or importance.
Clinical significance asks if the effect is meaningful for patients and changes clinical practice. It's the "so what?" test.
- Assessed using the Minimal Clinically Important Difference (MCID): the smallest change in a treatment outcome that a patient would identify as important.

⭐ A study with a massive sample size can show a statistically significant result (e.g., a tiny drop in blood pressure) that is too small to be clinically meaningful.

Clinical vs. statistical significance in disability decline

Statistical significance (p-value < 0.05) does not equal clinical significance.

Effect size (e.g., Cohen's d, odds ratio) measures the magnitude of a difference, informing clinical relevance.

Large sample sizes can yield statistically significant results for clinically meaningless effects.

Confidence intervals help assess both statistical significance and the precision of the effect estimate.

Clinical significance is a judgment call, considering the effect's magnitude, risks, benefits, and costs.

Continue reading on Oncourse

CONTINUE READING — FREE

or get the app

App Store|Google Play