Hypothesis Testing

On this page

Hypothesis Basics - Null's Game

  • Null Hypothesis (H₀): Statement of no effect or no difference; the hypothesis to be tested. E.g., $H_0: \mu_1 = \mu_2$.
  • Alternative Hypothesis (H₁): Statement that contradicts H₀; claims an effect or difference.
    • One-tailed test: Specifies direction (e.g., $H_1: \mu_1 > \mu_2$ or $H_1: \mu_1 < \mu_2$).
    • Two-tailed test: Does not specify direction (e.g., $H_1: \mu_1 \neq \mu_2$).
  • Level of Significance (α): Probability of rejecting H₀ when it is true (Type I error). Commonly α = 0.05.
  • Critical Region: Set of test statistic values for which H₀ is rejected. Determined by α. Null and Alternate Distributions with Alpha and Beta Errors and alternative hypothesis (H1) distributions, highlighting alpha level, critical region for one-tailed and two-tailed tests)

⭐ The null hypothesis always states there is no difference or no effect between the groups being compared or no association between the variables being studied.

Error Types & Power - Alpha's Oops, Beta's Blues

FeatureType I Error (α)Type II Error (β)
DefinitionRejecting a true H₀ (Null Hypothesis)Failing to reject a false H₀
A.k.a.False PositiveFalse Negative
Probability$P(\text{Type I error}) = \alpha$$P(\text{Type II error}) = \beta$
Mnemonic 📌Innocent man jailedGuilty man freed
-   Formula: Power = $1 - \beta$.
-   Ideally > **0.80**.

Type I and Type II Error Probability Distributions

  • Key Relationships:
    • α & β inversely related (fixed n): ↓α → ↑β.
    • ↑Sample Size (n) → ↓β (↑Power).
    • ↑Effect Size → ↓β (↑Power).

⭐ Decreasing α reduces Type I error chance but increases Type II error (β) chance, unless sample size (n) increases.

P-Value & Decisions - The Verdict Value

  • P-value: Probability of obtaining current test results, or more extreme, if the null hypothesis (H₀) is true. Measures strength of evidence against H₀.

  • Decision Rule ($\alpha$ = significance level, usually 0.05):

    • 📌 Mnemonic: If P is low, H₀ must go! (Reject H₀ if $p \leq \alpha$)
  • P-value & 95% Confidence Interval (CI) Relationship:

    • CI: Range of plausible values for a population parameter.

    ⭐ If a 95% CI for a difference does not include 0, or for a ratio does not include 1 (null values), then $p < \textbf{0.05}$ (statistically significant).

    • Conversely, if the 95% CI includes the null value, $p > \textbf{0.05}$_

Test Selection - Choosing Wisely

Test choice hinges on data type, distribution, and sample traits.

  • Parametric Tests: For numerical data following a normal distribution.
    • Key assumptions: Normality, homogeneity of variances, independence.
    • Examples:
      • t-test: Compares means between one/two groups.
        • One-sample t-test: Compares one group's mean to a known value.
        • Unpaired (Independent) t-test: Compares means of two distinct, unrelated groups.
        • Paired t-test: Compares means from one group at two times or matched pairs.
      • ANOVA (Analysis of Variance): Compares means of three or more independent groups.
  • Non-Parametric Tests: Used if parametric assumptions unmet, or for ordinal/nominal data.
    • Examples:
      • Chi-square test ($\chi^2$)*: Categorical data: tests association or goodness-of-fit.
      • Mann-Whitney U test: Compares two independent groups (alt. to unpaired t-test).
      • Wilcoxon signed-rank test: Compares two related/paired samples (alt. to paired t-test).
      • Kruskal-Wallis test: Compares three or more independent groups (alt. to ANOVA).

⭐ The Chi-square test is versatile for categorical data, used to assess if there's a significant association between two categorical variables or if observed frequencies differ from expected frequencies (goodness-of-fit).

High‑Yield Points - ⚡ Biggest Takeaways

  • Null Hypothesis (H0) proposes no difference or no relationship between variables.
  • Alternative Hypothesis (H1) suggests a significant difference or relationship exists.
  • Type I error (α) is rejecting a true H0 (false positive); its probability is the p-value.
  • Type II error (β) is failing to reject a false H0 (false negative).
  • Power of a study (1-β) is the probability of correctly detecting an effect if it exists.
  • The level of significance (α), the threshold for p-value, is commonly 0.05.
  • If p-value < α, reject H0; results are deemed statistically significant.

Practice Questions: Hypothesis Testing

Test your understanding with these related questions

A group of 80 people is being studied to determine the effect of diet modification on cholesterol levels. To compare the mean cholesterol levels before and after the diet modification in this group, which statistical test should be used?

1 of 5

Flashcards: Hypothesis Testing

1/10

_____ is also called as post-test probability of a disease/ precision rate

TAP TO REVEAL ANSWER

_____ is also called as post-test probability of a disease/ precision rate

PPV

browseSpaceflip

Enjoying this lesson?

Get full access to all lessons, practice questions, and more.

Start Your Free Trial