Tests of Significance

Hypothesis Testing - Null's Big Gamble

• Null Hypothesis ($H_0$): No difference/effect (e.g., no drug effect).
• Alternative Hypothesis ($H_A$): Difference/effect exists (e.g., drug has effect).
• Errors:
  • Type I Error ($\alpha$): Rejecting true $H_0$ (False Positive). Prob = $\alpha$. 📌 Mistake of commission.
    • $\alpha$ (Significance Level): e.g., 0.05, 0.01.
  • Type II Error ($\beta$): Failing to reject false $H_0$ (False Negative). Prob = $\beta$. 📌 Mistake of omission.
• Power: $1 - \beta$. Probability of detecting a true effect.
• P-value: Probability of observed data (or more extreme) if $H_0$ is true.
  • P $\le \alpha \implies$ Reject $H_0$ (Significant).
  • P $> \alpha \implies$ Fail to reject $H_0$ (Not Significant). 

⭐ P-value: Probability of current findings (or more extreme) if $H_0$ is true. Small p ($\le$ 0.05) = strong evidence against $H_0$.

Parametric Tests - Gaussian Gladiators

• Assumptions: Normal distribution, homogeneity of variances (Levene's test), interval/ratio scale, independent observations.
• Student's t-test: Compares means. $t = \frac{\text{signal}}{\text{noise}}$.
  • One-sample: Sample mean vs. population mean. $t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}}$.
  • Independent two-sample (Unpaired): Means of 2 independent groups.
  • Paired: Means of 1 group (2 times) or matched pairs. $t = \frac{\bar{d}}{s_d/\sqrt{n}}$.
• ANOVA (Analysis of Variance):
  • Compares means of ≥3 groups.
  • Uses F-statistic: $F = \frac{\text{variance between groups}}{\text{variance within groups}}$.
  • Types: One-way, Two-way.

⭐ ANOVA compares means of three or more groups. F-statistic: ratio of between-group to within-group variance.

Non-Parametric Tests - Skew Savvy Squad

• Use for skewed, ordinal data, or when parametric assumptions fail.
• Distribution-free: no population distribution assumptions.
• 📌 Mnemonic: Data Skewed, Ordinal, or Small? SOS! Non-parametric call!

Choosing Your Weapon - Test Selection Tactics

Selecting the appropriate statistical test is paramount for valid conclusions. Your choice hinges on understanding your data's nature and the study design. Ask these key questions first:

⭐ Key questions to ask: Type of data? Number of groups? Independent or paired samples? Data distribution?

Key decision factors:

• 1. Type of Data: Quantitative (numerical values like BP, HbA1c) or Qualitative (categories like gender, outcome)?
• 2. Number of Groups: Comparing two groups (e.g., cases vs. controls) or more than two groups?
• 3. Sample Relationship: Independent samples (e.g., two different patient groups) or Paired samples (e.g., same patient before/after treatment)?
• 4. Data Distribution (Quantitative Data): Normal (Gaussian) distribution → Parametric tests. Skewed distribution → Non-parametric tests.

High‑Yield Points - ⚡ Biggest Takeaways

• p-value ≤ 0.05 implies significance, leading to rejection of Null Hypothesis (H0).
• Null Hypothesis (H0): No difference. Alternative Hypothesis (H1): Difference exists.
• Type I error (α): Rejecting true H0 (false positive); P(Type I error) = α (significance level).
• Type II error (β): Failing to reject false H0 (false negative); Power = 1 - β.
• Student's t-test: Compares means of two groups (quantitative data, small samples).
• Chi-square test (χ²): For categorical data, tests association or proportions.
• ANOVA: Compares means of >2 groups (more than two groups).