P-values and confidence intervals

P-values and confidence intervals

P-values and confidence intervals

On this page

P-Values - Judging The Numbers

  • Definition: The probability of obtaining the observed study results, or more extreme results, if the null hypothesis (H₀) were true.
  • A small p-value suggests the observed data is unlikely if H₀ is true.
  • Alpha (α): The pre-set significance level. It's the threshold for rejecting H₀. Conventionally set at < 0.05.
  • If p < α (e.g., p < 0.05):
    • Result is statistically significant.
    • We reject the null hypothesis.
    • There is a <5% chance the result is due to random chance alone.
  • If p ≥ α (e.g., p ≥ 0.05):
    • Result is not statistically significant.
    • We fail to reject the null hypothesis.

⭐ A p-value does not indicate the size or clinical importance of an effect. A very large sample size can lead to a small p-value for a trivial effect.

Normal distribution with rejection and non-rejection regions

Confidence Intervals - The Range Game

  • A Confidence Interval (CI) is a range of values that likely contains the true population parameter (e.g., mean). It quantifies the uncertainty around an estimate.
  • Formula: $CI = \text{Point Estimate} \pm \text{Margin of Error}$.
  • Interpretation: For a 95% CI, we are 95% confident the true population value falls within that range.
  • CI Width & Precision:
    • Wider CI → Less precise estimate.
    • Narrower CI → More precise estimate.
    • Affected by: ↑ sample size → ↓ CI width; ↑ confidence level (99% vs 95%) → ↑ CI width.

Confidence intervals for average TV hours in UK vs USA

  • Statistical Significance:
    • For mean differences, if the CI includes 0, the result is NOT statistically significant (p ≥ 0.05).
    • For ratios (Odds Ratio, Relative Risk), if the CI includes 1, the result is NOT statistically significant (p ≥ 0.05).

⭐ If the 95% CI for an intervention does not cross the null value (0 for difference, 1 for ratio), the finding is statistically significant with a p-value < 0.05.

Errors & Power - Dodging Pitfalls

  • Type I Error (α): The "false positive." You reject a true null hypothesis (H₀).

    • Probability = α (significance level, e.g., 0.05).
    • 📌 You see an effect that isn't there.
  • Type II Error (β): The "false negative." You fail to reject a false null hypothesis (H₀).

    • Probability = β.
    • 📌 You are blind to an effect that is there.
  • Power (1 - β): Probability of detecting a true effect (correctly rejecting a false H₀).

    • To ↑ Power: ↑ sample size (n), ↑ effect size, or ↑ α.

Type I and Type II Errors in Hypothesis Testing

⭐ Power is typically set to 80% in clinical trials. This means investigators accept a 20% chance of a Type II error (β = 0.20), a common trade-off for feasibility.

High‑Yield Points - ⚡ Biggest Takeaways

  • A p-value is the probability of observing data if the null hypothesis is true.
  • If p < 0.05, results are statistically significant, and you reject the null hypothesis.
  • A confidence interval (CI) provides a range of plausible values for the true population parameter.
  • If a 95% CI for a mean difference excludes 0, or for a ratio excludes 1, the result is significant.
  • Narrow CIs indicate high precision, often from a larger sample size.
  • The p-value cutoff (α) is the probability of a Type I error.

Practice Questions: P-values and confidence intervals

Test your understanding with these related questions

Group of 100 medical students took an end of the year exam. The mean score on the exam was 70%, with a standard deviation of 25%. The professor states that a student's score must be within the 95% confidence interval of the mean to pass the exam. Which of the following is the minimum score a student can have to pass the exam?

1 of 5

Flashcards: P-values and confidence intervals

1/10

In the united states, the majority of overdose deaths are caused by _____

TAP TO REVEAL ANSWER

In the united states, the majority of overdose deaths are caused by _____

opioids

browseSpaceflip

Enjoying this lesson?

Get full access to all lessons, practice questions, and more.

Start Your Free Trial