You are conducting a study comparing the efficacy of two different statin medications. Two groups are placed on different statin medications, statin A and statin B. Baseline LDL levels are drawn for each group and are subsequently measured every 3 months for 1 year. Average baseline LDL levels for each group were identical. The group receiving statin A exhibited an 11 mg/dL greater reduction in LDL in comparison to the statin B group. Your statistical analysis reports a p-value of 0.052. Which of the following best describes the meaning of this p-value?

There is a 5.2% chance of observing a difference in reduction of LDL of 11 mg/dL or greater even if the two medications have identical effects

There is a 95% chance that the difference in reduction of LDL observed reflects a real difference between the two groups

Though A is more effective than B, there is a 5% chance the difference in reduction of LDL between the two groups is due to chance

If 100 permutations of this experiment were conducted, 5 of them would show similar results to those described above

This is a statistically significant result

A researcher is conducting a study to compare fracture risk in male patients above the age of 65 who received annual DEXA screening to peers who did not receive screening. He conducts a randomized controlled trial in 900 patients, with half of participants assigned to each experimental group. The researcher ultimately finds similar rates of fractures in the two groups. He then notices that he had forgotten to include 400 patients in his analysis. Including the additional participants in his analysis would most likely affect the study's results in which of the following ways?

Increased probability of rejecting the null hypothesis when it is truly false

Wider confidence intervals of results

Increased probability of committing a type II error

Decreased significance level of results

Increased external validity of results

You submit a paper to a prestigious journal about the effects of coffee consumption on mesothelioma risk. The first reviewer lauds your clinical and scientific acumen, but expresses concern that your study does not have adequate statistical power. Statistical power refers to which of the following?

The probability of detecting an association when an association does exist.

The probability of detecting an association when no association exists.

The probability of not detecting an association when an association does exist.

The square root of the variance.

A health system implements a new sepsis protocol across 20 hospitals. A researcher plans to evaluate effectiveness using a stepped-wedge cluster randomized design where hospitals sequentially adopt the protocol every 3 months. She calculates sample size based on individual patient outcomes (mortality) needing 2,000 patients total. The biostatistician identifies a critical error. Evaluate what modification is needed.

Account for intra-cluster correlation coefficient (ICC) requiring substantial sample size inflation

Adjust for multiple time periods using Bonferroni correction

Use hospital-level outcomes instead of patient-level outcomes as unit of analysis

Increase alpha to 0.10 to account for cluster randomization reducing power

Include random effects for both hospital and time period in power calculation

Factors affecting power - Free USMLE Review

Factors Affecting Power - The Core Quartet

Statistical Power: The probability of correctly rejecting a false null hypothesis (H₀), i.e., detecting a true effect. It is the complement of a Type II error (Power = $1 - \beta$).

Factor	Impact of ↑ Increase on Power	Rationale
Significance Level ($\alpha$)	↑ Power	↑ $\alpha$ (e.g., from 0.01 to 0.05) creates a larger rejection region, making it easier to find a significant result.
Sample Size ($n$)	↑ Power	A larger sample provides a more precise estimate of the population parameter, reducing standard error.
Effect Size ($\Delta$)	↑ Power	A larger difference or stronger relationship is inherently easier to detect.
Variability ($\sigma$)	↓ Power	↑ standard deviation leads to more overlap between distributions, obscuring the true effect.

Power & Error Rates - A Balancing Act

Power is the probability of correctly rejecting a false null hypothesis (detecting a true effect).
It's directly related to the Type II error rate ($\beta$): $Power = 1 - \beta$.
- $\beta$ is the probability of a false negative (failing to detect a true effect).
- A lower $\beta$ leads to a higher power.
The Trade-Off: Power is inversely affected by the Type I error rate ($\alpha$).
- $\alpha$ is the probability of a false positive (rejecting a true null hypothesis), usually set at 0.05.
- To be more confident in our result (↓ $\alpha$), we must accept a higher chance of missing a true effect (↑ $\beta$), which in turn ↓ Power.
- ↓ $\alpha$ → ↑ $\beta$ → ↓ Power.

Alpha, Beta, and Power in Hypothesis Testing

⭐ By convention, an acceptable $\beta$ is often set at 0.2, which corresponds to a power of 0.8 (or 80%). This means researchers accept a 20% chance of missing a real effect to achieve 80% power.

Sample Size Formula - Power by the Numbers

A simplified formula shows how different factors influence the required sample size ($n$):

$n \propto \frac{(\sigma^2)(Z_\alpha + Z_\beta)^2}{(\mu_1 - \mu_2)^2}$

$n$ (Sample Size): Number of subjects needed.
$\sigma$ (Standard Deviation): Data variability. As $\sigma$ ↑, $n$ ↑.
$Z_\alpha$ (Significance): Z-score for the chosen $\alpha$ level (e.g., 0.05). As $\alpha$ ↓, $n$ ↑.
$Z_\beta$ (Power): Z-score for the desired power (1 - $\beta$). As power ↑, $n$ ↑.
$\mu_1 - \mu_2$ (Effect Size): Magnitude of the difference to be detected. As effect size ↓, $n$ ↑.

⭐ To detect an effect size that is half as large, you must quadruple the sample size.

High‑Yield Points - ⚡ Biggest Takeaways

Power is the probability of detecting a true effect and avoiding a Type II error (Power = 1 - β).

↑ Sample size (n) is the most common way to increase power.

↑ Effect size (the magnitude of the difference) makes it easier to find a statistically significant result.

↑ Significance level (α) increases power, but also increases the risk of a Type I error.

↓ Variability (standard deviation) in the data increases power.