Effect size estimation

Effect Size Estimation - Sizing Up Significance

• Core Concept: Magnitude vs. Probability

  • Effect Size (ES): A crucial statistical measure that quantifies the magnitude of a phenomenon. It answers the question, "How much of an impact did the intervention have?" or "How strong is the relationship between these variables?"
  • This contrasts sharply with the p-value, which only indicates the probability of observing the study results if the null hypothesis were true. A p-value addresses "Is there a statistically significant effect?" not "How big is the effect?"
  • Clinical Relevance: A study can have a statistically significant result (e.g., p = 0.01) but a small effect size, meaning the finding is unlikely due to chance but may be too small to be clinically meaningful. Conversely, a large effect size might be clinically important even if it doesn't reach statistical significance, perhaps due to a small sample size (low power).

• Common Measures of Effect Size

  • • For Differences Between Two Group Means (Continuous Data):
    • • Cohen's d: The standardized difference between two means, expressed in units of standard deviation. It is the most common measure for comparing means from two groups (e.g., treatment vs. placebo).
    • • Formula: $d = \frac{{\text{Mean}{\text{group1}} - \text{Mean}{\text{group2}}}}{\text{Pooled Standard Deviation}}$
    • • Interpretation:
      • $d \approx \textbf{0.2}$: Small effect
      • $d \approx \textbf{0.5}$: Medium effect
      • $d \approx \textbf{0.8}$: Large effect
  • • For Proportions & Categorical Data (2x2 Tables):
    • • Odds Ratio (OR) & Relative Risk (RR): Quantify the strength of association between an exposure and an outcome.
    • • An OR/RR of 1.0 signifies no effect (the "null" value).
    • • The further the value is from 1.0, the larger the effect size. For example, an RR of 3.0 (risk is tripled) is a larger effect than an RR of 1.5 (risk is 50% higher).
  • • For Association Between Two Continuous Variables:
    • • Pearson's Correlation Coefficient (r): Measures the strength and direction of a linear relationship.
    • • Ranges from -1 to +1. The sign indicates direction (positive/negative), while the absolute value indicates strength.
    • • Interpretation (by absolute value |r|):
      • $|r| \approx \textbf{0.1}$: Small/weak correlation
      • $|r| \approx \textbf{0.3}$: Medium/moderate correlation
      • $|r| \approx \textbf{0.5}$: Large/strong correlation

• Flowchart: Integrating P-Value and Effect Size

• 

⭐ A very large sample size can make a tiny, clinically irrelevant effect statistically significant (e.g., p < 0.001). Always inspect the effect size (e.g., Cohen's d, OR/RR) to judge clinical importance. This is a classic trap in interpreting large-scale trial data.

High‑Yield Points - ⚡ Biggest Takeaways

• Effect size quantifies the magnitude of an intervention's effect or the strength of a relationship between variables.
• It is a crucial determinant of statistical power; a larger effect size requires a smaller sample size to achieve adequate power.
• Unlike p-value, effect size is independent of sample size and helps assess the practical or clinical significance of findings.
• Common measures include Cohen's d for mean differences and odds ratio/relative risk for proportions.