Hypothesis Testing

Hypothesis Basics - Null's Game

• Null Hypothesis (H₀): Statement of no effect or no difference; the hypothesis to be tested. E.g., $H_0: \mu_1 = \mu_2$.
• Alternative Hypothesis (H₁): Statement that contradicts H₀; claims an effect or difference.
  • One-tailed test: Specifies direction (e.g., $H_1: \mu_1 > \mu_2$ or $H_1: \mu_1 < \mu_2$).
  • Two-tailed test: Does not specify direction (e.g., $H_1: \mu_1 \neq \mu_2$).
• Level of Significance (α): Probability of rejecting H₀ when it is true (Type I error). Commonly α = 0.05.
• Critical Region: Set of test statistic values for which H₀ is rejected. Determined by α. and alternative hypothesis (H1) distributions, highlighting alpha level, critical region for one-tailed and two-tailed tests)

⭐ The null hypothesis always states there is no difference or no effect between the groups being compared or no association between the variables being studied.

Error Types & Power - Alpha's Oops, Beta's Blues

-   Formula: Power = $1 - \beta$.
-   Ideally > **0.80**.

• Key Relationships:
  • α & β inversely related (fixed n): ↓α → ↑β.
  • ↑Sample Size (n) → ↓β (↑Power).
  • ↑Effect Size → ↓β (↑Power).

⭐ Decreasing α reduces Type I error chance but increases Type II error (β) chance, unless sample size (n) increases.

P-Value & Decisions - The Verdict Value

• P-value: Probability of obtaining current test results, or more extreme, if the null hypothesis (H₀) is true. Measures strength of evidence against H₀.

• Decision Rule ($\alpha$ = significance level, usually 0.05):

  • 📌 Mnemonic: If P is low, H₀ must go! (Reject H₀ if $p \leq \alpha$)

• P-value & 95% Confidence Interval (CI) Relationship:

  • CI: Range of plausible values for a population parameter.

  ⭐ If a 95% CI for a difference does not include 0, or for a ratio does not include 1 (null values), then $p < \textbf{0.05}$ (statistically significant).

  • Conversely, if the 95% CI includes the null value, $p > \textbf{0.05}$_

Test Selection - Choosing Wisely

Test choice hinges on data type, distribution, and sample traits.

• Parametric Tests: For numerical data following a normal distribution.
  • Key assumptions: Normality, homogeneity of variances, independence.
  • Examples:
    • t-test: Compares means between one/two groups.
      • One-sample t-test: Compares one group's mean to a known value.
      • Unpaired (Independent) t-test: Compares means of two distinct, unrelated groups.
      • Paired t-test: Compares means from one group at two times or matched pairs.
    • ANOVA (Analysis of Variance): Compares means of three or more independent groups.
• Non-Parametric Tests: Used if parametric assumptions unmet, or for ordinal/nominal data.
  • Examples:
    • Chi-square test ($\chi^2$)*: Categorical data: tests association or goodness-of-fit.
    • Mann-Whitney U test: Compares two independent groups (alt. to unpaired t-test).
    • Wilcoxon signed-rank test: Compares two related/paired samples (alt. to paired t-test).
    • Kruskal-Wallis test: Compares three or more independent groups (alt. to ANOVA).

⭐ The Chi-square test is versatile for categorical data, used to assess if there's a significant association between two categorical variables or if observed frequencies differ from expected frequencies (goodness-of-fit).

High‑Yield Points - ⚡ Biggest Takeaways

• Null Hypothesis (H0) proposes no difference or no relationship between variables.
• Alternative Hypothesis (H1) suggests a significant difference or relationship exists.
• Type I error (α) is rejecting a true H0 (false positive); its probability is the p-value.
• Type II error (β) is failing to reject a false H0 (false negative).
• Power of a study (1-β) is the probability of correctly detecting an effect if it exists.
• The level of significance (α), the threshold for p-value, is commonly 0.05.
• If p-value < α, reject H0; results are deemed statistically significant.