Confidence intervals for non-parametric tests

Non-Parametric CIs - Beyond the Bell Curve

• Used when data is skewed or does not follow a normal distribution, violating assumptions of parametric tests (like t-tests).
• Confidence intervals are constructed for the median, not the mean.
• Common Methods:
  • Wilcoxon Rank-Sum Test: The CI for the difference between two medians is based on the Hodges-Lehmann estimator, which uses the median of all possible pairwise differences between the two samples.
  • Sign Test: The CI for a single sample median is derived from the binomial distribution.
  • Bootstrapping: A resampling technique to estimate CIs for any statistic without distribution assumptions.

⭐ High-Yield: Bootstrapping is a versatile, computer-intensive method. It involves repeatedly sampling with replacement from the data to create thousands of new samples, generating a distribution of the statistic (e.g., median) to find the confidence interval.

Wilcoxon & Mann-Whitney - Sizing Up Medians

• Use Case: Essential for skewed data or when assumptions for t-tests (like normal distribution) are not met. These tests analyze medians, not means.

• Mann-Whitney U Test (Wilcoxon Rank-Sum Test)

  • Compares medians of two independent groups.
  • Analogy: Non-parametric version of the independent samples t-test.
  • Example: Comparing the median symptom score in a treatment group vs. a placebo group.

• Wilcoxon Signed-Rank Test

  • Compares medians of two paired/matched samples.
  • Analogy: Non-parametric version of the paired t-test.
  • Example: Evaluating median cholesterol levels in the same patients before and after a new diet.

⭐ Power Consideration: While robust, non-parametric tests are less powerful than their parametric counterparts (t-tests) if the data actually follows a normal distribution. This means a higher risk of a Type II error (failing to detect a real effect).

Wilcoxon Signed-Rank - Paired Data Power

• Analogue: Non-parametric equivalent to the paired t-test.
• Use When:
  • Data is from paired samples (e.g., before-and-after measurements).
  • Data is not normally distributed.
  • Outcome is ordinal or continuous.
• Mechanism:
  • Calculates the difference for each pair: $d_i = \text{after}_i - \text{before}_i$.
  • Ranks the absolute values of non-zero differences.
  • Sums the ranks for positive differences ($W^+$) and negative differences ($W^-$).
  • The test statistic, $W$, is the smaller of $W^+$ and $W^-$.
• Power: More powerful than the Sign Test; less powerful than the paired t-test if data is normally distributed.

⭐ High-Yield Pearl: The Wilcoxon Signed-Rank test retains approximately 95% of the statistical power of the paired t-test even on data that perfectly fits the t-test's normal distribution assumptions, making it an excellent and safe alternative.

High‑Yield Points - ⚡ Biggest Takeaways

• Non-parametric CIs estimate the population median, not the mean, as they are based on ranks.
• The CI for a Wilcoxon test estimates the median of the difference between groups (independent or paired).
• Essential for skewed data or samples with outliers, where the mean is not a reliable measure.
• Interpretation is key: if a 95% CI for a difference in medians contains 0, the result is not significant.
• Generally wider (less precise) than parametric CIs because they use less data information.