Bayesian alternatives to p-values

On this page

P-Value Problems - The Frequentist's Flaws

  • Arbitrary Cutoff: Relies on a fixed threshold (e.g., p < 0.05), a convention, not a constant of nature.
  • Ignores Effect Size: A tiny, clinically irrelevant effect can be statistically significant with a large sample size.
  • Prone to "P-hacking": Manipulating data or statistical tests to achieve a desired significant p-value.

⭐ A p-value is the probability of observing the current data (or more extreme) if the null hypothesis were true. It is NOT the probability of the null hypothesis being true.

Bayesian Basics - Updating Your Beliefs

  • Core Idea: Updates prior beliefs (prior probability) with new evidence (likelihood) to form an updated belief (posterior probability). It asks: "How likely is my hypothesis, given the data?"
  • Bayes' Theorem: $P( ext{Hypothesis}|\text{Data}) = \frac{P(\text{Data}|\text{Hypothesis}) \times P(\text{Hypothesis})}{P(\text{Data})}$
    • Prior: Your belief before seeing the data.
    • Likelihood: The evidence from your study.
    • Posterior: Your updated belief.
  • Contrasts with frequentist methods, which don't incorporate prior beliefs.

High-Yield: The choice of the prior can significantly influence the posterior probability. A strong prior requires more compelling evidence to shift the belief.

Bayes Factor - The Evidence Scale

  • The Bayes Factor (BF) quantifies how much more likely the data are under one hypothesis compared to another. It directly compares the evidence for the alternative hypothesis (H₁) versus the null hypothesis (H₀).
  • Formula: $BF_{10} = \frac{P(\text{Data}|H_1)}{P(\text{Data}|H_0)}$

| Bayes Factor (BF₁₀) & Strength of Evidence | | :--- | :--- | | > 100 | Extreme evidence for H₁ | | 30 - 100 | Very Strong evidence for H₁ | | 10 - 30 | Strong evidence for H₁ | | 3 - 10 | Moderate evidence for H₁ | | 1 - 3 | Anecdotal evidence for H₁ |> ⭐ Unlike p-values, a BF < 1 (e.g., 1/10) provides evidence for the null hypothesis, suggesting the data are more probable under H₀.

Bayes factors vs. p-values across sample sizes

Credible Intervals - A More Believable Range

  • A Bayesian alternative to the frequentist confidence interval (CI).
  • Represents a range of values where the true parameter has a certain probability of lying, given the data and prior beliefs.
  • Derived from the posterior probability distribution (prior beliefs + data likelihood).
  • For a 95% credible interval, one can state there is a 95% probability that the true value of the parameter is within this interval.

⭐ The key advantage is its direct, intuitive interpretation, which is how many mistakenly interpret frequentist confidence intervals.

Frequentist vs. Bayesian - The Final Showdown

FeatureFrequentist ApproachBayesian Approach
Core IdeaProbability as long-run frequency. Parameters are fixed constants.Probability as a degree of belief. Parameters are random variables.
Key MetricP-value: Pr(data | H₀ true). If p < 0.05, reject H₀.Bayes Factor (BF): Compares evidence for H₁ vs. H₀.
Interval95% Confidence Interval: Captures true parameter in 95% of repeated samples.95% Credible Interval: 95% probability the true parameter lies in this range.
InputData only.Data + Prior belief.
OutputP-values & CIs.Posterior probability distributions.

High‑Yield Points - ⚡ Biggest Takeaways

  • Bayesian inference updates prior probability with new data to create a posterior probability.
  • The Bayes factor (BF) is the ratio of the likelihood of two competing hypotheses; it quantifies evidence.
  • A BF > 1 indicates evidence for the alternative hypothesis; a BF < 1 supports the null hypothesis.
  • Unlike p-values, Bayes factors can provide direct evidence for the null hypothesis, not just a failure to reject it.
  • This avoids the common fallacy of treating non-significance as proof of no effect.

Practice Questions: Bayesian alternatives to p-values

Test your understanding with these related questions

A scientist in Chicago is studying a new blood test to detect Ab to EBV with increased sensitivity and specificity. So far, her best attempt at creating such an exam reached 82% sensitivity and 88% specificity. She is hoping to increase these numbers by at least 2 percent for each value. After several years of work, she believes that she has actually managed to reach a sensitivity and specificity much greater than what she had originally hoped for. She travels to China to begin testing her newest blood test. She finds 2,000 patients who are willing to participate in her study. Of the 2,000 patients, 1,200 of them are known to be infected with EBV. The scientist tests these 1,200 patients' blood and finds that only 120 of them tested negative with her new exam. Of the patients who are known to be EBV-free, only 20 of them tested positive. Given these results, which of the following correlates with the exam's specificity?

1 of 5

Flashcards: Bayesian alternatives to p-values

1/8

Accuracy may also be referred to as _____

TAP TO REVEAL ANSWER

Accuracy may also be referred to as _____

validity

browseSpaceflip

Enjoying this lesson?

Get full access to all lessons, practice questions, and more.

Start For Free