Tests of Significance

On this page

Hypothesis Testing - Null's Big Gamble

  • Null Hypothesis ($H_0$): No difference/effect (e.g., no drug effect).
  • Alternative Hypothesis ($H_A$): Difference/effect exists (e.g., drug has effect).
  • Errors:
    • Type I Error ($\alpha$): Rejecting true $H_0$ (False Positive). Prob = $\alpha$. 📌 Mistake of commission.
      • $\alpha$ (Significance Level): e.g., 0.05, 0.01.
    • Type II Error ($\beta$): Failing to reject false $H_0$ (False Negative). Prob = $\beta$. 📌 Mistake of omission.
  • Power: $1 - \beta$. Probability of detecting a true effect.
  • P-value: Probability of observed data (or more extreme) if $H_0$ is true.
    • P $\le \alpha \implies$ Reject $H_0$ (Significant).
    • P $> \alpha \implies$ Fail to reject $H_0$ (Not Significant). ![Error Matrix](type one two error matrix)

⭐ P-value: Probability of current findings (or more extreme) if $H_0$ is true. Small p ($\le$ 0.05) = strong evidence against $H_0$.

Parametric Tests - Gaussian Gladiators

  • Assumptions: Normal distribution, homogeneity of variances (Levene's test), interval/ratio scale, independent observations.
  • Student's t-test: Compares means. $t = \frac{\text{signal}}{\text{noise}}$.
    • One-sample: Sample mean vs. population mean. $t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}}$.
    • Independent two-sample (Unpaired): Means of 2 independent groups.
    • Paired: Means of 1 group (2 times) or matched pairs. $t = \frac{\bar{d}}{s_d/\sqrt{n}}$.
  • ANOVA (Analysis of Variance):
    • Compares means of ≥3 groups.
    • Uses F-statistic: $F = \frac{\text{variance between groups}}{\text{variance within groups}}$.
    • Types: One-way, Two-way.

⭐ ANOVA compares means of three or more groups. F-statistic: ratio of between-group to within-group variance.

Non-Parametric Tests - Skew Savvy Squad

  • Use for skewed, ordinal data, or when parametric assumptions fail.
  • Distribution-free: no population distribution assumptions.
  • 📌 Mnemonic: Data Skewed, Ordinal, or Small? SOS! Non-parametric call!
Non-Parametric TestParametric CounterpartUse
Chi-square Test ($\chi^2$)-Categorical data (nominal/ordinal); tests independence/goodness-of-fit. $\chi^2 = \sum \frac{(O-E)^2}{E}$
Fisher's Exact Test-2x2 tables with small samples (any expected cell count <5)
Mann-Whitney U TestIndependent t-test2 independent groups (ordinal/non-normal); compares medians
Wilcoxon Signed-Rank TestPaired t-test2 paired samples (ordinal/non-normal); compares medians, considers magnitude
Kruskal-Wallis TestOne-way ANOVA3 independent groups (ordinal/non-normal); compares medians

Choosing Your Weapon - Test Selection Tactics

Selecting the appropriate statistical test is paramount for valid conclusions. Your choice hinges on understanding your data's nature and the study design. Ask these key questions first:

⭐ Key questions to ask: Type of data? Number of groups? Independent or paired samples? Data distribution?

Key decision factors:

  • 1. Type of Data: Quantitative (numerical values like BP, HbA1c) or Qualitative (categories like gender, outcome)?
  • 2. Number of Groups: Comparing two groups (e.g., cases vs. controls) or more than two groups?
  • 3. Sample Relationship: Independent samples (e.g., two different patient groups) or Paired samples (e.g., same patient before/after treatment)?
  • 4. Data Distribution (Quantitative Data): Normal (Gaussian) distribution → Parametric tests. Skewed distribution → Non-parametric tests.

Parametric vs Non-Parametric Tests Comparison

High‑Yield Points - ⚡ Biggest Takeaways

  • p-value ≤ 0.05 implies significance, leading to rejection of Null Hypothesis (H0).
  • Null Hypothesis (H0): No difference. Alternative Hypothesis (H1): Difference exists.
  • Type I error (α): Rejecting true H0 (false positive); P(Type I error) = α (significance level).
  • Type II error (β): Failing to reject false H0 (false negative); Power = 1 - β.
  • Student's t-test: Compares means of two groups (quantitative data, small samples).
  • Chi-square test (χ²): For categorical data, tests association or proportions.
  • ANOVA: Compares means of >2 groups (more than two groups).

Practice Questions: Tests of Significance

Test your understanding with these related questions

After applying a statistical test, an investigator gets a p-value of 0.01. What does this indicate about the null hypothesis?

1 of 5

Flashcards: Tests of Significance

1/10

_____ is also called as post-test probability of a disease/ precision rate

TAP TO REVEAL ANSWER

_____ is also called as post-test probability of a disease/ precision rate

PPV

browseSpaceflip

Enjoying this lesson?

Get full access to all lessons, practice questions, and more.

Start Your Free Trial