P-values and confidence intervals — MCQs

Two research groups independently study the same genetic variant's association with diabetes. Study A (n=5,000) reports OR=1.25, 95% CI: 1.05-1.48, p=0.01. Study B (n=50,000) reports OR=1.08, 95% CI: 1.02-1.14, p=0.006. Both studies are methodologically sound. Synthesize these findings to determine the most likely true effect and evaluate implications for clinical and research interpretation.

A prestigious journal publishes a trial showing a new cancer drug extends survival by 2 months (p=0.001, 95% CI: 1.5-2.5 months). The drug costs $150,000 per patient and causes Grade 3-4 toxicity in 60% of patients. Three prior unpublished trials showed non-significant results (all p>0.20). Synthesize these findings to evaluate the evidence base.

A pharmaceutical company conducts 20 different analyses on their trial data, testing for effects on various secondary outcomes. One analysis shows a significant benefit (p=0.03) on hospital readmission rates. The primary outcome (mortality) showed p=0.12. The company seeks FDA approval based on the readmission data. Evaluate the validity and implications of this approach.

A study reports that a new diagnostic test has 95% sensitivity and 90% specificity for detecting coronary artery disease, with both confidence intervals excluding the performance of the current standard test. However, when analyzed by subgroups, the p-values for sensitivity are 0.001 in men but 0.45 in women, despite similar point estimates. Analyze what this pattern suggests.

A meta-analysis combines 15 studies on vitamin D supplementation and fracture risk. The pooled relative risk is 0.88 (95% CI: 0.79-0.98, p=0.02). However, individual studies showed p-values ranging from 0.10 to 0.85, with none reaching significance alone. Analyze the implications of this finding.

Two separate randomized trials evaluate the same antidepressant. Trial A (n=100) shows a 5-point improvement on a depression scale with p=0.06 and 95% CI of -0.2 to 10.2. Trial B (n=800) shows a 5-point improvement with p=0.001 and 95% CI of 3.5-6.5. Analyze why these trials yield different p-values despite identical point estimates.

A small pilot study (n=40) examines a new therapy for septic shock. Mortality is 25% in the treatment group versus 45% in the control group. The p-value is 0.08 and the 95% confidence interval for the absolute risk reduction is -2% to 42%. Apply these findings to determine next steps.

A pharmaceutical company tests a new cholesterol-lowering drug in 10,000 patients. The drug reduces LDL cholesterol by 2 mg/dL compared to placebo, with p<0.001 and 95% CI of 1.5-2.5 mg/dL. The company emphasizes the highly significant p-value in marketing materials. Apply critical evaluation to this scenario.

A cohort study evaluates the association between coffee consumption and risk of Parkinson's disease over 20 years. The hazard ratio is 0.70 with a 95% confidence interval of 0.55-0.89 and p=0.004. A colleague argues the results are not meaningful because the confidence interval is wide. Apply statistical reasoning to evaluate this interpretation.

Q10

A randomized controlled trial compares a new antihypertensive medication to placebo in 500 patients. After 6 months, the mean systolic blood pressure reduction is 12 mmHg in the treatment group versus 3 mmHg in the placebo group. The p-value is 0.03 and the 95% confidence interval for the difference is 2-18 mmHg. Apply these findings to clinical practice.

P-values and confidence intervals US Medical PG Practice Questions and MCQs

Question 1: Two research groups independently study the same genetic variant's association with diabetes. Study A (n=5,000) reports OR=1.25, 95% CI: 1.05-1.48, p=0.01. Study B (n=50,000) reports OR=1.08, 95% CI: 1.02-1.14, p=0.006. Both studies are methodologically sound. Synthesize these findings to determine the most likely true effect and evaluate implications for clinical and research interpretation.

A. Study B is definitive because of its larger sample size and should replace Study A's findings
B. The study with the lower p-value (Study B) is automatically more reliable
C. The studies are contradictory and no conclusions can be drawn
D. Study A is correct because it was published first
E. The true effect is likely modest (closer to Study B's estimate); Study A likely overestimated due to smaller sample size, but both show statistical significance with clinically marginal effects (Correct Answer)

Explanation: ***The true effect is likely modest (closer to Study B's estimate); Study A likely overestimated due to smaller sample size, but both show statistical significance with clinically marginal effects*** - Study B has significantly higher **statistical power** and **precision** (narrower 95% CI) due to its larger sample size, making its **odds ratio (OR)** estimate more reliable. - Smaller initial studies often exhibit the **Winner's Curse**, where effect sizes are **overestimated** to reach the threshold for statistical significance. *Study A is correct because it was published first* - **Publication order** does not determine the scientific validity or accuracy of genetic association studies. - Early studies are more prone to **random error** and inflated effect sizes compared to later, larger-scale replications. *Study B is definitive because of its larger sample size and should replace Study A's findings* - While Study B is more **precise**, both studies are directionally consistent and both show **statistical significance** (p < 0.05). - Scientific evidence is **cumulative**; Study B refines and confirms the existence of an association rather than declaring Study A's findings as entirely false. *The studies are contradictory and no conclusions can be drawn* - The studies are not contradictory because both **confidence intervals** show an OR > 1.0, and both reach **statistical significance**. - Both groups found the same **direction of effect**, suggesting a real albeit modest genetic association with diabetes. *The study with the lower p-value (Study B) is automatically more reliable* - Reliability depends on **methodological rigor** and **precision**, whereas the p-value is heavily influenced by **sample size**. - A lower p-value indicates stronger evidence against the **null hypothesis** but does not inherently mean the study is free from bias or more reliable in its effect estimate.

Question 2: A prestigious journal publishes a trial showing a new cancer drug extends survival by 2 months (p=0.001, 95% CI: 1.5-2.5 months). The drug costs $150,000 per patient and causes Grade 3-4 toxicity in 60% of patients. Three prior unpublished trials showed non-significant results (all p>0.20). Synthesize these findings to evaluate the evidence base.

A. This pattern suggests publication bias; the significant result may be a false positive among multiple trials, and the modest benefit must be weighed against substantial toxicity and cost (Correct Answer)
B. The confidence interval proves the drug should be standard of care
C. P-values below 0.01 override concerns about prior negative studies
D. The published study's highly significant p-value validates the drug's efficacy
E. The three unpublished trials are irrelevant to evaluating the published study

Explanation: ***This pattern suggests publication bias; the significant result may be a false positive among multiple trials, and the modest benefit must be weighed against substantial toxicity and cost*** - The existence of three unpublished negative trials alongside one positive one strongly indicates **publication bias** (the file drawer effect), suggesting the positive result might be a **Type I error** or an overestimation. - **Statistical significance** (p=0.001) does not equal **clinical significance**; a marginal 2-month survival gain must be balanced against extreme **financial cost** and a 60% rate of **Grade 3-4 toxicity**. *The published study's highly significant p-value validates the drug's efficacy* - A **low p-value** only indicates that the null hypothesis is unlikely within that specific trial; it does not account for the **context** of other failed experiments. - Efficacy cannot be validated in isolation when the broader **evidence base** (including unpublished data) shows inconsistent results. *The three unpublished trials are irrelevant to evaluating the published study* - All relevant clinical trials must be synthesized via **meta-analysis** or systematic review to determine the true **effect size** of an intervention. - Ignoring unpublished data leads to **evidence distortion**, where clinicians perceive a drug as more effective than it truly is. *P-values below 0.01 override concerns about prior negative studies* - No **p-value** can magically override the **prior probability** of a drug's success; consistent negative results in prior trials increase the likelihood that a later positive result is a **false positive**. - High-impact medical decisions require a consistent **body of evidence** rather than a single outlier result, regardless of the level of significance. *The confidence interval proves the drug should be standard of care* - The **95% Confidence Interval** (1.5–2.5 months) tells us only about the **precision** of the measurement, not the **magnitude of clinical benefit**. - Becoming a **standard of care** requires a favorable **risk-benefit ratio**, which is undermined here by severe **adverse events** and poor **cost-effectiveness**.

Question 3: A pharmaceutical company conducts 20 different analyses on their trial data, testing for effects on various secondary outcomes. One analysis shows a significant benefit (p=0.03) on hospital readmission rates. The primary outcome (mortality) showed p=0.12. The company seeks FDA approval based on the readmission data. Evaluate the validity and implications of this approach.

A. Secondary outcomes are more important than primary outcomes when significant
B. The p=0.03 result is valid and supports approval regardless of the primary outcome
C. Any p<0.05 in a clinical trial justifies approval
D. This represents multiple testing without correction, inflating Type I error; the significant result may be due to chance and selective reporting (Correct Answer)
E. The mortality p-value of 0.12 is close enough to significance to support both findings

Explanation: ***This represents multiple testing without correction, inflating Type I error; the significant result may be due to chance and selective reporting*** - Performing **multiple comparisons** (20 analyses) without adjustment increases the probability of a **false positive** result; by chance alone, 1 out of 20 tests is expected to be significant at p < 0.05. - Reliable conclusions require **post-hoc corrections** (like Bonferroni) or pre-specified hierarchies to prevent **selective reporting** or "p-hacking" of secondary outcomes. *The p=0.03 result is valid and supports approval regardless of the primary outcome* - A result is not considered valid in isolation when it is one of many tests; the **Type I error rate** is not maintained at 5%. - Regulatory approval usually requires the **primary outcome** to be met, as secondary outcomes are generally considered **hypothesis-generating**. *Secondary outcomes are more important than primary outcomes when significant* - **Primary outcomes** are the pre-defined measures that the trial is specifically powered to detect; ignoring them leads to **bias**. - Significance in a **secondary outcome** cannot supersede a non-significant primary outcome, especially when the test wasn't protected against multiple comparisons. *The mortality p-value of 0.12 is close enough to significance to support both findings* - In frequentist statistics, a **p-value of 0.12** is greater than the standard threshold of 0.05 and must be interpreted as **not statistically significant**. - "Close" results do not validate other weak findings; they suggest the study failed to reject the **null hypothesis** for the most important clinical endpoint. *Any p<0.05 in a clinical trial justifies approval* - Approval requires evidence of both **statistical significance** and **clinical relevance**, typically demonstrated in the primary endpoint. - **Spurious correlations** occur frequently in large datasets; therefore, a single p < 0.05 obtained through **data dredging** is insufficient for regulatory standards.

Question 4: A study reports that a new diagnostic test has 95% sensitivity and 90% specificity for detecting coronary artery disease, with both confidence intervals excluding the performance of the current standard test. However, when analyzed by subgroups, the p-values for sensitivity are 0.001 in men but 0.45 in women, despite similar point estimates. Analyze what this pattern suggests.

A. Women were enrolled incorrectly in the study
B. P-values below 0.05 in any subgroup validate the overall finding
C. The test is only valid for men and should not be used in women
D. The test should be rejected because of inconsistent significance
E. The difference in p-values likely reflects unequal sample sizes between subgroups rather than true performance differences (Correct Answer)

Explanation: ***The difference in p-values likely reflects unequal sample sizes between subgroups rather than true performance differences*** - When **point estimates** (e.g., sensitivity) are nearly identical across subgroups but **p-values** vary significantly, it often indicates a difference in **statistical power**. - A high p-value in one group (women) despite high sensitivity suggests the **sample size** for that subgroup was too small to reject the **null hypothesis**. *The test is only valid for men and should not be used in women* - Validity is determined by the **point estimate** of diagnostic accuracy; a non-significant p-value due to small sample size does not mean the test is ineffective. - Clinical utility should be based on **effect size** and confidence intervals; rejecting a test solely on a **subgroup p-value** is a common statistical error. *P-values below 0.05 in any subgroup validate the overall finding* - Subgroup significance does not automatically validate the **primary endpoint** or generalizability to the entire population. - Each subgroup analysis must be interpreted in the context of **multiplicity** and the study's overall **statistical design**. *The test should be rejected because of inconsistent significance* - Significance is a function of both **effect size** and **n (sample size)**; inconsistency in p-values without a change in point estimates is a result of study design, not test failure. - The **confidence intervals** for the overall test performance excluded the standard test, suggesting the new test is likely superior despite the subgroup limitation. *Women were enrolled incorrectly in the study* - Lack of statistical significance does not imply **procedural errors** or incorrect enrollment of participants. - This pattern typically points to **under-enrollment** of a specific demographic rather than a flaw in the inclusion/exclusion criteria or clinical methodology.

Question 5: A meta-analysis combines 15 studies on vitamin D supplementation and fracture risk. The pooled relative risk is 0.88 (95% CI: 0.79-0.98, p=0.02). However, individual studies showed p-values ranging from 0.10 to 0.85, with none reaching significance alone. Analyze the implications of this finding.

A. The significant pooled result represents increased power from combining studies, but heterogeneity and publication bias should be evaluated (Correct Answer)
B. A significant meta-analysis with non-significant component studies indicates data manipulation
C. The meta-analysis p-value is the only value that matters
D. The meta-analysis is invalid because none of the individual studies were significant
E. Individual study p-values should be averaged rather than pooled

Explanation: ***The significant pooled result represents increased power from combining studies, but heterogeneity and publication bias should be evaluated*** - Combining studies increases the **sample size**, which enhances the **statistical power** to detect small but clinically relevant differences that individual studies may be underpowered to find. - Even if the result is significant, it is essential to assess **heterogeneity** (consistency between studies) and **publication bias** (missing negative data) to ensure the pooled estimate is reliable. *The meta-analysis is invalid because none of the individual studies were significant* - A meta-analysis does not require individual studies to be significant; its primary purpose is to aggregate data to overcome **Type II errors** found in smaller trials. - The validity of a meta-analysis depends on the **methodological quality** of the included studies and the appropriateness of the **pooling techniques**, not the p-values of single trials. *The meta-analysis p-value is the only value that matters* - While the p-value indicates **statistical significance**, the **95% Confidence Interval** (0.79-0.98) is equally important as it provides the range of the expected effect size. - Focusing solely on the p-value ignores the **clinical significance** and the potential impact of **systematic biases** inherent in meta-analytic summaries. *Individual study p-values should be averaged rather than pooled* - Averaging p-values is statistically incorrect because p-values are not linear; instead, meta-analysis pools the **raw data** or **effect sizes** (like Relative Risk) to calculate a weighted average. - Weighting is typically based on the **inverse of the variance**, giving more influence to larger, more precise studies rather than a simple arithmetic average. *A significant meta-analysis with non-significant component studies indicates data manipulation* - This scenario is a standard and expected outcome of meta-analyses, often referred to as resolving **stochastic noise** to find a true underlying signal. - Significant pooled results from non-significant studies are the hallmark of **increased precision** and do not serve as evidence of **data fabrication** or unethical practices.

Question 6: Two separate randomized trials evaluate the same antidepressant. Trial A (n=100) shows a 5-point improvement on a depression scale with p=0.06 and 95% CI of -0.2 to 10.2. Trial B (n=800) shows a 5-point improvement with p=0.001 and 95% CI of 3.5-6.5. Analyze why these trials yield different p-values despite identical point estimates.

A. Trial A used incorrect statistical methods
B. The difference reflects sample size; Trial B has greater power to detect the same effect, yielding a lower p-value and narrower confidence interval (Correct Answer)
C. Trial A's result is actually more significant because it approached significance with fewer patients
D. Trial B's result is more reliable because it has a lower p-value
E. The p-values are incomparable between different sized studies

Explanation: ***The difference reflects sample size; Trial B has greater power to detect the same effect, yielding a lower p-value and narrower confidence interval*** - A larger **sample size** (n=800 vs n=100) reduces the **standard error**, producing a narrower **95% confidence interval** (3.5-6.5) that excludes the null value. - Increased **statistical power** in Trial B allows for the detection of a **statistically significant** difference (p=0.001), whereas Trial A is **underpowered** to reach significance (p=0.06). *Trial A used incorrect statistical methods* - There is no evidence of **methodological error**; the difference in results is purely a function of **mathematical precision** related to sample size. - Both trials reported identical **point estimates**, suggesting the calculation of the mean improvement was handled consistently. *Trial B's result is more reliable because it has a lower p-value* - A lower **p-value** indicates **statistical significance**, but "reliability" in a clinical sense refers to the **reproducibility** and **precision** of the estimate. - While Trial B is more precise due to the **narrower confidence interval**, p-values alone do not determine the quality or reliability of a study's design. *Trial A's result is actually more significant because it approached significance with fewer patients* - Approaching significance (p=0.06) does not grant **statistical significance**; the null hypothesis cannot be rejected if the p-value exceeds **alpha (0.05)**. - This interpretation is a common **misconception**; statistical significance is a binary threshold and Trial A failed to cross it. *The p-values are incomparable between different sized studies* - **P-values** are directly comparable as they provide a standardized measure of the probability that the observed effect occurred by **random chance**. - However, they must be interpreted alongside **sample size** and **effect size** to understand why the levels of significance differ between the studies.

Question 7: A small pilot study (n=40) examines a new therapy for septic shock. Mortality is 25% in the treatment group versus 45% in the control group. The p-value is 0.08 and the 95% confidence interval for the absolute risk reduction is -2% to 42%. Apply these findings to determine next steps.

A. The confidence interval proves the therapy is beneficial
B. The results are promising but inconclusive; the confidence interval includes both harm and substantial benefit, warranting a larger trial (Correct Answer)
C. The results justify immediate clinical implementation
D. The p-value of 0.08 is close enough to 0.05 to claim significance
E. The therapy is ineffective because p>0.05

Explanation: ***The results are promising but inconclusive; the confidence interval includes both harm and substantial benefit, warranting a larger trial*** - A **p-value of 0.08** is greater than the standard alpha of 0.05, meaning the results are **not statistically significant** and could be due to chance. - The **95% confidence interval** ranges from -2% (harm) to 42% (benefit); because it **crosses the null value (0)**, it indicates the study is **underpowered** due to the small sample size (n=40). *The therapy is ineffective because p>0.05* - A p-value greater than 0.05 does not prove a therapy is **ineffective**; it only means the study failed to reject the **null hypothesis**. - The large observed difference (20% mortality reduction) suggests a potential benefit that requires a **larger sample size** to confirm or refute. *The results justify immediate clinical implementation* - Implementation is premature because the results failed to reach **statistical significance**, meaning the observed benefit may not be reproducible. - Clinical guidelines require robust evidence through **Phase III trials** with sufficient power before a new therapy for septic shock becomes standard of care. *The confidence interval proves the therapy is beneficial* - A confidence interval only proves benefit if it does not contain the **null value** (zero for risk reduction or one for risk ratios). - Since this interval includes **-2%**, it suggests a possibility that the treatment could actually **increase mortality** (harm), prohibiting a claim of proven benefit. *The p-value of 0.08 is close enough to 0.05 to claim significance* - In frequentist statistics, a pre-defined **alpha (usually 0.05)** is a strict threshold; values above this are categorized as **non-significant**. - "Trend towards significance" or "close enough" are common pitfalls in interpretation that do not substitute for **statistical rigor**.

Question 8: A pharmaceutical company tests a new cholesterol-lowering drug in 10,000 patients. The drug reduces LDL cholesterol by 2 mg/dL compared to placebo, with p<0.001 and 95% CI of 1.5-2.5 mg/dL. The company emphasizes the highly significant p-value in marketing materials. Apply critical evaluation to this scenario.

A. Statistical significance always indicates clinical significance
B. The p-value is more important than the confidence interval for clinical decisions
C. The p<0.001 confirms this is an important clinical finding
D. The narrow confidence interval proves the drug is effective
E. The result is statistically significant but clinically trivial; large sample size produced a significant p-value for a minimal effect (Correct Answer)

Explanation: ***The result is statistically significant but clinically trivial; large sample size produced a significant p-value for a minimal effect*** - A very large **sample size (n=10,000)** increases the **statistical power**, allowing the study to detect even minor differences as statistically significant (**p<0.001**). - While the result is unlikely due to chance, an **LDL reduction of 2 mg/dL** is clinically insignificant and would not meaningfully change a patient's **cardiovascular risk profile**. *The p<0.001 confirms this is an important clinical finding* - A low **p-value** only indicates that the null hypothesis can be rejected; it does not measure the **clinical impact** or importance of the treatment. - **Clinical importance** is determined by the **effect size** and patient outcomes, which are minimal in this specific scenario. *The narrow confidence interval proves the drug is effective* - A **narrow confidence interval (1.5-2.5 mg/dL)** indicates high **precision** of the estimate, but it precisely confirms that the effect is consistently small. - Efficiency and **effectiveness** in clinical terms require the magnitude of the change to meet a therapeutic threshold, which this drug fails to do. *The p-value is more important than the confidence interval for clinical decisions* - **Confidence intervals** provide more information than p-values because they show the **magnitude and direction** of the effect, which is vital for clinical decision-making. - **Clinical decisions** should be based on whether the **effect size** (e.g., 2 mg/dL) justifies the cost, side effects, and usage of the medication. *Statistical significance always indicates clinical significance* - **Statistical significance** is a mathematical threshold, whereas **clinical significance** is a judgment of whether the results are meaningful to the patient. - This scenario is a classic example of how a **large study population** can produce a significant p-value for a result that has no **practical medical utility**.

Question 9: A cohort study evaluates the association between coffee consumption and risk of Parkinson's disease over 20 years. The hazard ratio is 0.70 with a 95% confidence interval of 0.55-0.89 and p=0.004. A colleague argues the results are not meaningful because the confidence interval is wide. Apply statistical reasoning to evaluate this interpretation.

A. The colleague is correct; wide confidence intervals indicate unreliable results
B. The confidence interval width is irrelevant when p<0.05
C. The entire confidence interval indicates protective effect, and the p-value confirms statistical significance (Correct Answer)
D. A confidence interval this wide requires a p-value less than 0.001 for significance
E. The results should be rejected because the confidence interval does not include 1.0

Explanation: ***The entire confidence interval indicates protective effect, and the p-value confirms statistical significance*** - Statistical significance is achieved because the **95% Confidence Interval (CI)** (0.55-0.89) does not cross the **null value of 1.0**, indicating a consistently lower risk in the coffee group. - A **p-value of 0.004** is less than the standard alpha of 0.05, reinforcing that the observed **30% reduction in risk** (Hazard Ratio 0.70) is unlikely to be due to chance. *The colleague is correct; wide confidence intervals indicate unreliable results* - While a wide CI suggests lower **precision**, it does not inherently mean the result is unreliable if it remains entirely on one side of the **null value**. - The result is highly **statistically significant** because the entire range (0.55 to 0.89) suggests a **protective effect** against the disease. *The confidence interval width is irrelevant when p<0.05* - CI width is never irrelevant as it provides information about the **precision of the estimate** and the range of possible effect sizes. - While the **p-value** tells us there is an effect, the **CI** tells us about the magnitude and certainty of that effect, providing more clinical context. *The results should be rejected because the confidence interval does not include 1.0* - A CI that **does not include 1.0** is precisely the criterion for **rejecting the null hypothesis** in studies using ratios (Hazard Ratio, Relative Risk, Odds Ratio). - If the CI included 1.0, it would mean there is no statistically significant difference between the **exposed and unexposed** groups. *A confidence interval this wide requires a p-value less than 0.001 for significance* - The threshold for **statistical significance** (usually p < 0.05) is independent of the width of the CI; they are mathematically related, but there is no "p < 0.001" requirement for wider intervals. - The **p-value of 0.004** already provides strong evidence against the null hypothesis, regardless of the **precision** measured by the CI width.

Question 10: A randomized controlled trial compares a new antihypertensive medication to placebo in 500 patients. After 6 months, the mean systolic blood pressure reduction is 12 mmHg in the treatment group versus 3 mmHg in the placebo group. The p-value is 0.03 and the 95% confidence interval for the difference is 2-18 mmHg. Apply these findings to clinical practice.

A. The results are statistically significant and the confidence interval suggests a meaningful clinical effect (Correct Answer)
B. The p-value indicates the probability that the null hypothesis is true
C. The results would be significant only if p<0.01
D. The confidence interval is too wide to make any clinical conclusions
E. The results are not clinically significant because the confidence interval includes values less than 10 mmHg

Explanation: ***The results are statistically significant and the confidence interval suggests a meaningful clinical effect*** - A **p-value of 0.03** is less than the standard threshold of 0.05, and the **95% confidence interval (2-18 mmHg)** does not include zero, confirming **statistical significance**. - The **net reduction of 9 mmHg** (12 minus 3) is a clinically relevant magnitude that is associated with a reduction in **cardiovascular morbidity and mortality**. *The results are not clinically significant because the confidence interval includes values less than 10 mmHg* - Clinical significance is determined by whether the **point estimate** and a portion of the **confidence interval** represent a meaningful impact on patient health, not just a subjective threshold like 10 mmHg. - Even small reductions in systolic blood pressure, as represented by the lower bound of **2 mmHg**, can contribute to **cumulative risk reduction** in population health. *The p-value indicates the probability that the null hypothesis is true* - A **p-value** is the probability of observing the results (or more extreme results) assuming the **null hypothesis is actually true**, not the probability that the hypothesis itself is true. - This is a common **misinterpretation of p-values**; they measure the strength of evidence against the null hypothesis rather than its literal truth. *The confidence interval is too wide to make any clinical conclusions* - A **confidence interval** that does not cross the null (zero for differences) is precise enough to determine **directionality and significance** of the treatment effect. - While a narrower interval provides more **precision**, an interval of **2-18 mmHg** still confirms that the new medication is superior to the placebo. *The results would be significant only if p<0.01* - The standard convention for **statistical significance** in most medical research is an **alpha level of 0.05**, making a p-value of 0.03 significant. - Requiring **p < 0.01** is a more stringent threshold (often used for multiple comparisons) but is not the default requirement for a standard **randomized controlled trial**.

Practice by Chapter

Definition and interpretation of p-values

Practice Questions

Common misinterpretations of p-values

Practice Questions

Confidence interval construction

Practice Questions

Relationship between CIs and hypothesis testing

Practice Questions

Multiple comparison problem

Practice Questions

Correction methods (Bonferroni, FDR)

Practice Questions

One-sided vs two-sided tests

Practice Questions

Clinical vs statistical significance

Practice Questions

Effect sizes and confidence intervals

Practice Questions

Bayesian alternatives to p-values

Practice Questions

Reporting standards in medical journals

Practice Questions

P-value controversy and alternatives

Practice Questions

Confidence intervals for non-parametric tests

Practice Questions

Want unlimited practice?

Get full access to all questions, explanations, and performance tracking.

Start For Free