Hypothesis testing

Hypothesis Testing - The Null Hypothesis Games

• Null Hypothesis ($H_0$): Assumes no difference or relationship (e.g., new drug is no better than placebo). It's the baseline assumption to be challenged.
• Alternative Hypothesis ($H_A$): Proposes a real difference or relationship exists.

• Errors in Decision:
  • Type I Error (α): Falsely rejecting a true $H_0$. 📌 Accusing an innocent person.
  • Type II Error (β): Failing to reject a false $H_0$. 📌 Letting a guilty person go free.
  • Power = $1 - β$. Probability of detecting a true effect.

⭐ The p-value is the probability of observing the study results (or more extreme) if the null hypothesis were actually true.

Error Analysis - When Good Tests Go Bad

• Type I Error ($α$): False Positive.

  • Incorrectly rejecting a true null hypothesis ($H_0$). You conclude there is a difference, when one doesn't exist.
  • The $p$-value represents the probability of making a Type I error.
  • Alpha ($α$) is the pre-set probability of making a Type I error, typically < 0.05.
  • 📌 Think: an innocent person is found guilty.

• Type II Error ($β$): False Negative.

  • Failing to reject a false null hypothesis ($H_0$). You conclude there is no difference, when one actually exists.
  • 📌 Think: a guilty person is set free.

• Power ($1-β$):

  • The probability of correctly detecting a true effect (correctly rejecting a false $H_0$).
  • Increase power by: ↑ sample size, ↑ effect size, or ↑ $α$ level.

⭐ The most common way to increase the power of a study is to increase the sample size.

Statistical Significance - Power & P-Values

• P-value: Probability of observing a result as or more extreme than the current one, assuming the null hypothesis (H₀) is true.
  • If p < α → Reject H₀ → Statistically significant result.
• Significance Level (α): Pre-specified probability of a Type I error. Standard threshold is α = 0.05.
• Confidence Interval (CI): Range of values likely to contain the true population value.
  • For mean difference: if CI does not include 0, result is significant.
  • For OR/RR: if CI does not include 1, result is significant.

• Power (1-β): Probability of correctly rejecting a false H₀ (detecting a true effect).
  • Factors that ↑ Power: ↑ Sample size, ↑ Effect size, ↑ α.

⭐ A 95% Confidence Interval that does not cross its null value (0 for difference, 1 for ratio) corresponds to a p-value < 0.05.

• Errors in Hypothesis Testing:
  • Type I Error (α): False positive. Rejecting a true H₀. 📌 Accusing an innocent person.
  • Type II Error (β): False negative. Failing to reject a false H₀. 📌 Blindingly letting a guilty person go free.

High‑Yield Points - ⚡ Biggest Takeaways

• The null hypothesis (H₀) assumes no effect or difference, while the alternative hypothesis (H₁) proposes one.
• A p-value is the probability of obtaining observed results, assuming the null hypothesis is true.
• If p ≤ α (typically 0.05), the result is statistically significant, and H₀ is rejected.
• Type I error (α) is rejecting a true null hypothesis (a false positive).
• Type II error (β) is failing to reject a false null hypothesis (a false negative).
• Power (1 - β) is the probability of detecting an effect when it exists.