Type I and Type II errors

Hypothesis Testing - The Core Concept

• Null Hypothesis ($H_0$): The default assumption that there is no relationship or difference between groups. E.g., a new drug has no effect.
• Alternative Hypothesis ($H_a$): Contradicts $H_0$; posits that a true relationship or difference exists.
• P-value: The probability of observing the study's findings (or more extreme) purely by chance, assuming $H_0$ is true.
• Alpha ($\[alpha\]$): The pre-set threshold for statistical significance, typically 0.05. It's the risk of a Type I error.

⭐ If the p-value is less than alpha, we reject the null hypothesis. This result is deemed "statistically significant."

Error Matrix - Two Ways to Be Wrong

• Type I Error (α): False Positive. Rejecting a true null hypothesis (H₀). You claim a difference exists when it does not.
  • The p-value represents the probability of committing a Type I error.
  • α is the pre-set threshold for significance, typically 0.05.
  • 📌 Think: Accusing an innocent person (Type A / I Error).
• Type II Error (β): False Negative. Failing to reject a false null hypothesis. You miss a difference that truly exists.
  • Power, the ability to detect a true effect, is calculated as $1 - β$.
  • 📌 Think: Being blind to a real difference (Type B / II Error).

⭐ Power ($1 - β$) is the probability of correctly identifying an effect when one exists. The most common way to increase power is to increase the sample size.

Power & Its Pals - Finding a Real Effect

Statistical power is the probability of detecting a true effect, avoiding a Type II error. It's the ability to correctly reject a false null hypothesis (H₀).

• Formula: Power = $1 - \beta$
• Goal: To have high power, typically ≥ 0.80.

Factors Influencing Power:

• Sample Size (n): ↑ n → ↑ Power
• Effect Size: ↑ difference between groups → ↑ Power
• Alpha Level (α): ↑ α → ↑ Power (but ↑ risk of Type I error)
• Standard Deviation (σ): ↓ σ (less variability) → ↑ Power

📌 Mnemonic: More Power with Plenty of People (large n) and a Palpable effect.

⭐ Most clinical trials aim for a power of 0.80, which means they accept a 20% chance of committing a Type II error (β). This is the accepted standard for finding a true effect.

High‑Yield Points - ⚡ Biggest Takeaways

• Type I error (α): A false-positive conclusion. You incorrectly reject a true null hypothesis (H₀).
• Type II error (β): A false-negative conclusion. You incorrectly fail to reject a false null hypothesis (H₀).
• Power (1 - β) is the probability of detecting a true effect. The most common way to increase power is to increase the sample size.
• The p-value is the probability of committing a Type I error. A result is significant if p < α (typically < 0.05).
• α and β have an inverse relationship; decreasing the risk of a Type I error increases the risk of a Type II error.