Hypothesis testing

On this page

Hypothesis Testing - The Null Hypothesis Games

  • Null Hypothesis ($H_0$): Assumes no difference or relationship (e.g., new drug is no better than placebo). It's the baseline assumption to be challenged.
  • Alternative Hypothesis ($H_A$): Proposes a real difference or relationship exists.
  • Errors in Decision:
    • Type I Error (α): Falsely rejecting a true $H_0$. 📌 Accusing an innocent person.
    • Type II Error (β): Failing to reject a false $H_0$. 📌 Letting a guilty person go free.
    • Power = $1 - β$. Probability of detecting a true effect.

⭐ The p-value is the probability of observing the study results (or more extreme) if the null hypothesis were actually true.

Type I and Type II Errors in Hypothesis Testing

Error Analysis - When Good Tests Go Bad

  • Type I Error ($α$): False Positive.

    • Incorrectly rejecting a true null hypothesis ($H_0$). You conclude there is a difference, when one doesn't exist.
    • The $p$-value represents the probability of making a Type I error.
    • Alpha ($α$) is the pre-set probability of making a Type I error, typically < 0.05.
    • 📌 Think: an innocent person is found guilty.
  • Type II Error ($β$): False Negative.

    • Failing to reject a false null hypothesis ($H_0$). You conclude there is no difference, when one actually exists.
    • 📌 Think: a guilty person is set free.
  • Power ($1-β$):

    • The probability of correctly detecting a true effect (correctly rejecting a false $H_0$).
    • Increase power by: ↑ sample size, ↑ effect size, or ↑ $α$ level.

Type I and Type II Errors in Hypothesis Testing

⭐ The most common way to increase the power of a study is to increase the sample size.

Statistical Significance - Power & P-Values

  • P-value: Probability of observing a result as or more extreme than the current one, assuming the null hypothesis (H₀) is true.
    • If p < α → Reject H₀ → Statistically significant result.
  • Significance Level (α): Pre-specified probability of a Type I error. Standard threshold is α = 0.05.
  • Confidence Interval (CI): Range of values likely to contain the true population value.
    • For mean difference: if CI does not include 0, result is significant.
    • For OR/RR: if CI does not include 1, result is significant.

Type I and Type II Error 2x2 Table

  • Power (1-β): Probability of correctly rejecting a false H₀ (detecting a true effect).
    • Factors that ↑ Power: ↑ Sample size, ↑ Effect size, ↑ α.

⭐ A 95% Confidence Interval that does not cross its null value (0 for difference, 1 for ratio) corresponds to a p-value < 0.05.

  • Errors in Hypothesis Testing:
    • Type I Error (α): False positive. Rejecting a true H₀. 📌 Accusing an innocent person.
    • Type II Error (β): False negative. Failing to reject a false H₀. 📌 Blindingly letting a guilty person go free.

High‑Yield Points - ⚡ Biggest Takeaways

  • The null hypothesis (H₀) assumes no effect or difference, while the alternative hypothesis (H₁) proposes one.
  • A p-value is the probability of obtaining observed results, assuming the null hypothesis is true.
  • If p ≤ α (typically 0.05), the result is statistically significant, and H₀ is rejected.
  • Type I error (α) is rejecting a true null hypothesis (a false positive).
  • Type II error (β) is failing to reject a false null hypothesis (a false negative).
  • Power (1 - β) is the probability of detecting an effect when it exists.

Practice Questions: Hypothesis testing

Test your understanding with these related questions

A research team develops a new monoclonal antibody checkpoint inhibitor for advanced melanoma that has shown promise in animal studies as well as high efficacy and low toxicity in early phase human clinical trials. The research team would now like to compare this drug to existing standard of care immunotherapy for advanced melanoma. The research team decides to conduct a non-randomized study where the novel drug will be offered to patients who are deemed to be at risk for toxicity with the current standard of care immunotherapy, while patients without such risk factors will receive the standard treatment. Which of the following best describes the level of evidence that this study can offer?

1 of 5

Flashcards: Hypothesis testing

1/10

Which stage of change is characterized by changing behaviors? _____

TAP TO REVEAL ANSWER

Which stage of change is characterized by changing behaviors? _____

Action/will power

browseSpaceflip

Enjoying this lesson?

Get full access to all lessons, practice questions, and more.

Start Your Free Trial