Post-hoc power analysis limitations US Medical PG Practice Questions and MCQs
Practice US Medical PG questions for Post-hoc power analysis limitations. These multiple choice questions (MCQs) cover important concepts and help you prepare for your exams.
Post-hoc power analysis limitations US Medical PG Question 1: A 25-year-old man with a genetic disorder presents for genetic counseling because he is concerned about the risk that any children he has will have the same disease as himself. Specifically, since childhood he has had difficulty breathing requiring bronchodilators, inhaled corticosteroids, and chest physiotherapy. He has also had diarrhea and malabsorption requiring enzyme replacement therapy. If his wife comes from a population where 1 in 10,000 people are affected by this same disorder, which of the following best represents the likelihood a child would be affected as well?
- A. 0.01%
- B. 2%
- C. 0.5%
- D. 1% (Correct Answer)
- E. 50%
Post-hoc power analysis limitations Explanation: ***Correct Option: 1%***
- The patient's symptoms (difficulty breathing requiring bronchodilators, inhaled corticosteroids, and chest physiotherapy; diarrhea and malabsorption requiring enzyme replacement therapy) are classic for **cystic fibrosis (CF)**, an **autosomal recessive disorder**.
- For an autosomal recessive disorder with a prevalence of 1 in 10,000 in the general population, **q² = 1/10,000**, so **q = 1/100 = 0.01**. The carrier frequency **(2pq)** is approximately **2q = 2 × (1/100) = 1/50 = 0.02**.
- The affected man is **homozygous recessive (aa)** and will always pass on the recessive allele. His wife has a **1/50 chance of being a carrier (Aa)**. If she is a carrier, she has a **1/2 chance of passing on the recessive allele**.
- Therefore, the probability of an affected child = **(Probability wife is a carrier) × (Probability wife passes recessive allele) = 1/50 × 1/2 = 1/100 = 1%**.
*Incorrect Option: 0.01%*
- This percentage is too low and does not correctly account for the carrier frequency in the population and the probability of transmission from a carrier mother.
*Incorrect Option: 2%*
- This represents approximately the carrier frequency (1/50 ≈ 2%), but does not account for the additional 1/2 probability that a carrier mother would pass on the recessive allele.
*Incorrect Option: 0.5%*
- This value would be correct if the carrier frequency were 1/100 instead of 1/50, which does not match the given population prevalence.
*Incorrect Option: 50%*
- **50%** would be the risk if both parents were carriers of an autosomal recessive disorder (1/4 chance = 25% for affected, but if we know one parent passes the allele, conditional probability changes). More accurately, 50% would apply if the disorder were **autosomal dominant** with one affected parent, which is not the case here.
Post-hoc power analysis limitations US Medical PG Question 2: A randomized control double-blind study is conducted on the efficacy of 2 sulfonylureas. The study concluded that medication 1 was more efficacious in lowering fasting blood glucose than medication 2 (p ≤ 0.05; 95% CI: 14 [10-21]). Which of the following is true regarding a 95% confidence interval (CI)?
- A. If the same study were repeated multiple times, approximately 95% of the calculated confidence intervals would contain the true population parameter. (Correct Answer)
- B. The 95% confidence interval is the probability chosen by the researcher to be the threshold of statistical significance.
- C. When a 95% CI for the estimated difference between groups contains the value ‘0’, the results are significant.
- D. It represents the probability that chance would not produce the difference shown, 95% of the time.
- E. The study is adequately powered at the 95% confidence interval.
Post-hoc power analysis limitations Explanation: ***If the same study were repeated multiple times, approximately 95% of the calculated confidence intervals would contain the true population parameter.***
- This statement accurately defines the **frequentist interpretation** of a confidence interval (CI). It reflects the long-run behavior of the CI over hypothetical repetitions of the study.
- A 95% CI means that if you were to repeat the experiment many times, 95% of the CIs calculated from those experiments would capture the **true underlying population parameter**.
*The 95% confidence interval is the probability chosen by the researcher to be the threshold of statistical significance.*
- The **alpha level (α)**, typically set at 0.05 (or 5%), is the threshold for statistical significance (p ≤ 0.05), representing the probability of a Type I error.
- The 95% confidence level (1-α) is related to statistical significance, but it is not the *threshold* itself; rather, it indicates the **reliability** of the interval estimate.
*When a 95% CI for the estimated difference between groups contains the value ‘0’, the results are significant.*
- If a 95% CI for the difference between groups **contains 0**, it implies that there is **no statistically significant difference** between the groups at the 0.05 alpha level.
- A statistically significant difference (p ≤ 0.05) would be indicated if the 95% CI **does NOT contain 0**, suggesting that the intervention had a real effect.
*It represents the probability that chance would not produce the difference shown, 95% of the time.*
- This statement misinterprets the meaning of a CI and probability. The chance of not producing the observed difference is typically addressed by the **p-value**, not directly by the CI in this manner.
- A CI provides a **range of plausible values** for the population parameter, not a probability about the role of chance in producing the observed difference.
*The study is adequately powered at the 95% confidence interval.*
- **Statistical power** is the probability of correctly rejecting a false null hypothesis, typically set at 80% or 90%. It is primarily determined by sample size, effect size, and alpha level.
- A 95% CI is a measure of the **precision** of an estimate, while power refers to the **ability of a study to detect an effect** if one exists. They are related but distinct concepts.
Post-hoc power analysis limitations US Medical PG Question 3: You are reading through a recent article that reports significant decreases in all-cause mortality for patients with malignant melanoma following treatment with a novel biological infusion. Which of the following choices refers to the probability that a study will find a statistically significant difference when one truly does exist?
- A. Type II error
- B. Type I error
- C. Confidence interval
- D. p-value
- E. Power (Correct Answer)
Post-hoc power analysis limitations Explanation: ***Power***
- **Power** is the probability that a study will correctly reject the null hypothesis when it is, in fact, false (i.e., will find a statistically significant difference when one truly exists).
- A study with high power minimizes the risk of a **Type II error** (failing to detect a real effect).
*Type II error*
- A **Type II error** (or **beta error**) occurs when a study fails to reject a false null hypothesis, meaning it concludes there is no significant difference when one actually exists.
- This is the **opposite** of what the question describes, which asks for the probability of *finding* a difference.
*Type I error*
- A **Type I error** (or **alpha error**) occurs when a study incorrectly rejects a true null hypothesis, concluding there is a significant difference when one does not actually exist.
- This relates to the **p-value** and the level of statistical significance (e.g., p < 0.05).
*Confidence interval*
- A **confidence interval** provides a range of values within which the true population parameter is likely to lie with a certain degree of confidence (e.g., 95%).
- It does not directly represent the probability of finding a statistically significant difference when one truly exists.
*p-value*
- The **p-value** is the probability of observing data as extreme as, or more extreme than, that obtained in the study, assuming the null hypothesis is true.
- It is used to determine statistical significance, but it is not the probability of detecting a true effect.
Post-hoc power analysis limitations US Medical PG Question 4: A medical research study is beginning to evaluate the positive predictive value of a novel blood test for non-Hodgkin’s lymphoma. The diagnostic arm contains 700 patients with NHL, of which 400 tested positive for the novel blood test. In the control arm, 700 age-matched control patients are enrolled and 0 are found positive for the novel test. What is the PPV of this test?
- A. 400 / (400 + 0) (Correct Answer)
- B. 700 / (700 + 300)
- C. 400 / (400 + 300)
- D. 700 / (700 + 0)
- E. 700 / (400 + 400)
Post-hoc power analysis limitations Explanation: ***400 / (400 + 0) = 1.0 or 100%***
- The **positive predictive value (PPV)** is calculated as **True Positives / (True Positives + False Positives)**.
- In this scenario, **True Positives (TP)** are the 400 patients with NHL who tested positive, and **False Positives (FP)** are 0, as no control patients tested positive.
- This gives a PPV of 400/400 = **1.0 or 100%**, indicating that all patients who tested positive actually had the disease.
*700 / (700 + 300)*
- This calculation does not align with the formula for PPV based on the given data.
- The denominator `(700+300)` suggests an incorrect combination of various patient groups.
*400 / (400 + 300)*
- The denominator `(400+300)` incorrectly includes 300, which is the number of **False Negatives** (patients with NHL who tested negative), not False Positives.
- PPV focuses on the proportion of true positives among all positive tests, not all diseased individuals.
*700 / (700 + 0)*
- This calculation incorrectly uses the total number of patients with NHL (700) as the numerator, rather than the number of positive test results in that group.
- The numerator should be the **True Positives** (400), not the total number of diseased individuals.
*700 / (400 + 400)*
- This calculation uses incorrect values for both the numerator and denominator, not corresponding to the PPV formula.
- The numerator 700 represents the total number of patients with the disease, not those who tested positive, and the denominator incorrectly sums up values that don't represent the proper PPV calculation.
Post-hoc power analysis limitations US Medical PG Question 5: You submit a paper to a prestigious journal about the effects of coffee consumption on mesothelioma risk. The first reviewer lauds your clinical and scientific acumen, but expresses concern that your study does not have adequate statistical power. Statistical power refers to which of the following?
- A. The probability of detecting an association when no association exists.
- B. The probability of not detecting an association when an association does exist.
- C. The probability of detecting an association when an association does exist. (Correct Answer)
- D. The first derivative of work.
- E. The square root of the variance.
Post-hoc power analysis limitations Explanation: ***The probability of detecting an association when an association does exist.***
- **Statistical power** is defined as the probability that a study will correctly reject a false null hypothesis, meaning it will detect a true effect or association if one exists.
- A study with **adequate statistical power** is less likely to miss a real effect.
*The probability of detecting an association when no association exists.*
- This describes a **Type I error** or **false positive**, often represented by **alpha (α)**.
- It is the probability of incorrectly concluding an effect or association exists when, in reality, there is none.
*The probability of not detecting an association when an association does exist.*
- This refers to a **Type II error** or **false negative**, represented by **beta (β)**.
- **Statistical power** is calculated as **1 - β**, so this option describes the complement of power.
*The first derivative of work.*
- The first derivative of work with respect to time represents **power** in physics, which is the rate at which work is done.
- This option is a **distractor** from physics and is unrelated to statistical power in research.
*The square root of the variance.*
- The **square root of the variance** is the **standard deviation**, a measure of the dispersion or spread of data.
- This is a statistical concept but is not the definition of statistical power.
Post-hoc power analysis limitations US Medical PG Question 6: A 35-year-old woman volunteers for a study on respiratory physiology. Pressure probes A and B are placed as follows:
Probe A: between the parietal and visceral pleura
Probe B: within the cavity of an alveolus
The probes provide a pressure reading relative to atmospheric pressure. To obtain a baseline reading, she is asked to sit comfortably and breathe normally. Which of the following sets of values will most likely be seen at the end of inspiration?
- A. Probe A: -6 mm Hg; Probe B: 0 mm Hg (Correct Answer)
- B. Probe A: 0 mm Hg; Probe B: -1 mm Hg
- C. Probe A: -4 mm Hg; Probe B: 0 mm Hg
- D. Probe A: -4 mm Hg; Probe B: -1 mm Hg
- E. Probe A: -6 mm Hg; Probe B: -1 mm Hg
Post-hoc power analysis limitations Explanation: ***Probe A: -6 mm Hg; Probe B: 0 mm Hg***
- At the **end of inspiration**, the **intrapleural pressure (Probe A)** is at its most negative, typically around -6 to -8 cm H2O (equivalent to -4 to -6 mmHg), reflecting the maximum expansion of the thoracic cavity.
- At the **end of inspiration**, just before exhalation begins, there is **no airflow**, so the **intrapulmonary pressure (Probe B)** equalizes with atmospheric pressure, resulting in a 0 mm Hg reading.
*Probe A: 0 mm Hg; Probe B: -1 mm Hg*
- An **intrapleural pressure of 0 mm Hg** would indicate a **pneumothorax** since it should always be negative to prevent lung collapse.
- An **intrapulmonary pressure of -1 mm Hg** would indicate that **inspiration is still ongoing**, as air would be flowing into the lungs.
*Probe A: -4 mm Hg; Probe B: 0 mm Hg*
- While an **intrapulmonary pressure of 0 mm Hg** is correct at the end of inspiration, an **intrapleural pressure of -4 mm Hg** is typical for the **end of expiration (Functional Residual Capacity)** during quiet breathing, not the end of inspiration.
- The **intrapleural pressure becomes more negative** during inspiration due to increased thoracic volume, so -4 mm Hg would be insufficient.
*Probe A: -4 mm Hg; Probe B: -1 mm Hg*
- An **intrapleural pressure of -4 mm Hg** is the normal pressure at the **end of expiration**, not the end of inspiration, where it becomes more negative.
- An **intrapulmonary pressure of -1 mm Hg** indicates that **inspiration is still in progress**, not at its end, as air would still be flowing into the lungs.
*Probe A: -6 mm Hg; Probe B: -1 mm Hg*
- While an **intrapleural pressure of -6 mm Hg** is consistent with the end of inspiration, an **intrapulmonary pressure of -1 mm Hg** means that **airflow is still occurring into the lungs**.
- At the **very end of inspiration**, just before the start of exhalation, airflow momentarily ceases, and intrapulmonary pressure becomes zero relative to the atmosphere.
Post-hoc power analysis limitations US Medical PG Question 7: The height of American adults is expected to follow a normal distribution, with a typical male adult having an average height of 69 inches with a standard deviation of 0.1 inches. An investigator has been informed about a community in the American Midwest with a history of heavy air and water pollution in which a lower mean height has been reported. The investigator plans to sample 30 male residents to test the claim that heights in this town differ significantly from the national average based on heights assumed be normally distributed. The significance level is set at 10% and the probability of a type 2 error is assumed to be 15%. Based on this information, which of the following is the power of the proposed study?
- A. 0.10
- B. 0.85 (Correct Answer)
- C. 0.90
- D. 0.15
- E. 0.05
Post-hoc power analysis limitations Explanation: ***0.85***
- **Power** is defined as **1 - β**, where β is the **probability of a Type II error**.
- Given that the probability of a **Type II error (β)** is 15% or 0.15, the power of the study is 1 - 0.15 = **0.85**.
*0.10*
- This value represents the **significance level (α)**, which is the probability of committing a **Type I error** (rejecting a true null hypothesis).
- The significance level is distinct from the **power of the study**, which relates to Type II errors.
*0.90*
- This value would be the power if the **Type II error rate (β)** was 0.10 (1 - 0.10 = 0.90), but the question specifies a β of 0.15.
- It is also the complement of the significance level (1 - α), which is not the definition of power.
*0.15*
- This value is the **probability of a Type II error (β)**, not the power of the study.
- **Power** is the probability of correctly rejecting a false null hypothesis, which is 1 - β.
*0.05*
- While 0.05 is a common significance level (α), it is not given as the significance level in this question (which is 0.10).
- This value also does not represent the power of the study, which would be calculated using the **Type II error rate**.
Post-hoc power analysis limitations US Medical PG Question 8: A health system implements a new sepsis protocol across 20 hospitals. A researcher plans to evaluate effectiveness using a stepped-wedge cluster randomized design where hospitals sequentially adopt the protocol every 3 months. She calculates sample size based on individual patient outcomes (mortality) needing 2,000 patients total. The biostatistician identifies a critical error. Evaluate what modification is needed.
- A. Adjust for multiple time periods using Bonferroni correction
- B. Use hospital-level outcomes instead of patient-level outcomes as unit of analysis
- C. Increase alpha to 0.10 to account for cluster randomization reducing power
- D. Include random effects for both hospital and time period in power calculation
- E. Account for intra-cluster correlation coefficient (ICC) requiring substantial sample size inflation (Correct Answer)
Post-hoc power analysis limitations Explanation: ***Account for intra-cluster correlation coefficient (ICC) requiring substantial sample size inflation***
- In cluster-randomized designs, observations within the same cluster (hospital) are not independent; the **Intra-cluster Correlation Coefficient (ICC)** quantifies this correlation and must be used to calculate a **design effect**.
- Neglecting the ICC leads to an **underpowered study** because the effective sample size is smaller than the total number of individual patients measured.
*Adjust for multiple time periods using Bonferroni correction*
- **Bonferroni correction** is used to control for **Type I error** when performing multiple independent hypothesis tests, not for determining sample size in nested longitudinal designs.
- While the stepped-wedge design involves multiple time points, the primary analysis typically uses a **single model** (e.g., GEE or GLMM) that accounts for time as a fixed effect.
*Use hospital-level outcomes instead of patient-level outcomes as unit of analysis*
- While the hospital is the **unit of randomization**, using hospital-level means as the unit of analysis simplifies the data and causes a significant loss of **statistical information** and precision.
- Modern biostatistical methods utilize **multilevel modeling** to maintain the richness of patient-level data while adjusting for the cluster-level randomization.
*Include random effects for both hospital and time period in power calculation*
- While random effects are important for the **analysis phase**, the "critical error" identified in the prompt refers to the initial failure to inflate the sample size based on **clustering (ICC)**.
- Power calculations for stepped-wedge designs are complex and certainly involve time parameters, but **ICC-based inflation** is the most fundamental adjustment required when moving from individual to cluster randomization.
*Increase alpha to 0.10 to account for cluster randomization reducing power*
- Increasing the **alpha level** (significance threshold) is not a standard or scientifically acceptable method to compensate for the loss of power due to **clustering**.
- Standard practice mandates maintaining an **alpha of 0.05** while appropriately increasing the **sample size** or number of clusters to reach the desired power (usually 80-90%).
Post-hoc power analysis limitations US Medical PG Question 9: A 41-year-old research fellow designs a non-inferiority trial comparing oral to IV antibiotics for osteomyelitis. She sets the non-inferiority margin at 10% (cure rate difference), expects 85% cure in both groups, and calculates 300 patients per arm for 80% power with α=0.025 (one-sided). Her mentor suggests this underestimates required sample size. Evaluate the mentor's concern.
- A. Correct; non-inferiority trials require larger samples than superiority trials for equivalent power (Correct Answer)
- B. Incorrect; non-inferiority trials actually require smaller samples due to less stringent hypotheses
- C. Correct; dropout rates in antibiotic trials necessitate 20% inflation of calculated sample size
- D. Incorrect; the calculation appropriately uses one-sided alpha for non-inferiority testing
- E. Correct; the margin should be set at 5% requiring doubling of sample size
Post-hoc power analysis limitations Explanation: ***Correct; non-inferiority trials require larger samples than superiority trials for equivalent power***
- **Non-inferiority trials** are designed to exclude a difference greater than a pre-specified margin, which typically requires a **larger sample size** than superiority trials investigating the same outcome.
- Because we are proving that the new treatment is "not much worse" (rather than "better"), the **statistical threshold** often necessitates higher enrollment to achieve adequate **power**.
*Incorrect; the calculation appropriately uses one-sided alpha for non-inferiority testing*
- While it is true that **non-inferiority testing** uses a **one-sided alpha (0.025)**, this does not negate the fact that such trials inherently require more participants.
- The mentor's concern is about the **total N**, which remains insufficient despite using the correct one-sided alpha convention.
*Correct; the margin should be set at 5% requiring doubling of sample size*
- There is no universal rule that the **non-inferiority margin** must be 5%; it is determined by **clinical significance** and regulatory standards for the specific condition.
- While a 5% margin would indeed increase the sample size, the 10% margin is often standard in **antibiotic trials** for osteomyelitis.
*Incorrect; non-inferiority trials actually require smaller samples due to less stringent hypotheses*
- This is a common misconception; non-inferiority trials are actually more demanding because the **null hypothesis** assumes the treatments are different (inferior).
- Disproving **inferiority** within a tight **margin (delta)** is statistically more intensive than proving a treatment is superior to a placebo.
*Correct; dropout rates in antibiotic trials necessitate 20% inflation of calculated sample size*
- While **attrition bias** is a concern, there is no fixed rule that every trial needs a **20% inflation** factor.
- The mentor's concern is specifically about the **base calculation** and the statistical nature of non-inferiority designs rather than just the **dropout rate**.
Post-hoc power analysis limitations US Medical PG Question 10: A pharmaceutical company tests a new antidepressant in 500 patients (250 per arm) and finds a 2-point improvement on a 52-point depression scale compared to placebo (p=0.04). The study was originally powered to detect a 4-point difference. The company seeks FDA approval citing statistical significance. Analyze the regulatory and scientific implications.
- A. Approval warranted; the study achieved statistical significance with adequate power
- B. Approval not warranted; observed effect is smaller than pre-specified clinically meaningful difference (Correct Answer)
- C. Approval warranted; post-hoc power analysis shows adequate power for 2-point difference
- D. Approval not warranted; the study was underpowered for the observed effect size
- E. Approval warranted if sensitivity analyses confirm robustness of findings
Post-hoc power analysis limitations Explanation: ***Approval not warranted; observed effect is smaller than pre-specified clinically meaningful difference***
- Although the result is **statistically significant** (p=0.04), the observed 2-point improvement is only half of the **pre-specified 4-point difference** deemed clinically relevant.
- Regulatory bodies like the **FDA** prioritize **clinical significance** over mere p-values, ensuring that a drug provides a meaningful benefit to patients' lives.
*Approval warranted; the study achieved statistical significance with adequate power*
- Statistical significance does not automatically justify approval if the **effect size** is too small to provide a real therapeutic advantage.
- Being **powered** for a 4-point difference means the study was designed to reliably detect a larger effect; a smaller effect may be a result of **over-testing** or limited clinical utility.
*Approval not warranted; the study was underpowered for the observed effect size*
- If a study finds a significant result (p < 0.05), it is by definition **sufficiently powered** to detect that specific effect size in that sample.
- The issue here is not **power** or sample size, but rather the **magnitude of effect** failing to meet the pre-defined target for clinical relevance.
*Approval warranted if sensitivity analyses confirm robustness of findings*
- **Sensitivity analyses** help confirm that results are not driven by outliers, but they cannot transform a **clinically trivial** difference into a meaningful one.
- Even a robust, consistent 2-point difference remains below the **Minimum Clinically Important Difference (MCID)** set at 4 points.
*Approval warranted; post-hoc power analysis shows adequate power for 2-point difference*
- **Post-hoc power analysis** is generally considered scientifically flawed and redundant once the **p-value** is already known.
- Demonstrating power for a 2-point difference does not erase the fact that the drug failed to meet the **threshold of efficacy** defined by the researchers at the start.
More Post-hoc power analysis limitations US Medical PG questions available in the OnCourse app. Practice MCQs, flashcards, and get detailed explanations.
