ROC curves

Sensitivity & Specificity - SNOUT, SPIN & Friends

• Sensitivity: Ability of a test to correctly identify individuals with the disease.
  • $Sensitivity = TP / (TP + FN)$
  • 📌 SNOUT: a highly Sensitive test, when Negative, rules OUT disease.
• Specificity: Ability of a test to correctly identify individuals without the disease.
  • $Specificity = TN / (TN + FP)$
  • 📌 SPIN: a highly Specific test, when Positive, rules IN disease.

⭐ Screening tests require high sensitivity to not miss cases (low false negatives). Confirmatory tests need high specificity to ensure those testing positive truly have the disease (low false positives).

ROC Curves - The Diagnostic Dance

• Function: Plots test performance across all cut-off points, visualizing the trade-off between sensitivity and specificity.
• Axes Definition:
  • Y-axis: True Positive Rate (TPR) = $Sensitivity$
  • X-axis: False Positive Rate (FPR) = $1 - Specificity$
• Interpretation:
  • The closer the curve hugs the top-left corner, the better the test.
  • A diagonal line ($y=x$) signifies a useless test (e.g., a coin flip).
• Area Under the Curve (AUC):
  • Represents overall test accuracy.
  • AUC = 1.0: Perfect test.
  • AUC = 0.5: No diagnostic value.

⭐ The ideal cut-off point on an ROC curve is often the one nearest to the top-left corner (0,1), as it maximizes both sensitivity and specificity. This point corresponds to the highest Youden's Index ($J = Sensitivity + Specificity - 1$).

Interpreting ROC & AUC - Curve Appeal & Power

• ROC Curve: Plots test sensitivity (TPR) vs. 1-Specificity (FPR) across all possible threshold values.
  • Y-axis: Sensitivity (True Positive Rate)
  • X-axis: 1 - Specificity (False Positive Rate)
• Area Under the Curve (AUC): Represents the overall diagnostic accuracy of a test. It measures the ability of the test to correctly distinguish between diseased and non-diseased individuals.
  • AUC = 1.0: Perfect test.
  • AUC = 0.5: Useless test (equivalent to a coin flip); represented by a diagonal line.
• Interpretation: The more the curve bends towards the top-left corner (↑ sensitivity, ↓ false positives), the higher the AUC and the better the test.

⭐ The AUC is equivalent to the probability that a randomly chosen diseased subject is ranked higher by the test than a randomly chosen healthy subject.

Choosing Cut-off Points - Picking Your Poison

• The chosen cut-off point for a diagnostic test creates a trade-off between sensitivity and specificity.
• Lowering the cut-off: ↑ Sensitivity, ↓ Specificity. Catches more cases, but also more false positives.
• Raising the cut-off: ↓ Sensitivity, ↑ Specificity. Misses more cases, but diagnoses are more certain.

⭐ For screening tests (e.g., initial HIV ELISA), a lower cut-off is preferred to maximize sensitivity (Rule OUT). For confirmatory tests (e.g., Western blot), a higher cut-off is used to maximize specificity (Rule IN).

• ROC curves plot Sensitivity (True Positive Rate) against 1-Specificity (False Positive Rate).
• The Area Under the Curve (AUC) is a measure of the overall accuracy of a diagnostic test.
• An AUC of 1.0 represents a perfect test, while an AUC of 0.5 (the diagonal line) signifies a useless test.
• The optimal cutoff point on the curve is typically closest to the top-left corner.
• A steeper curve indicates a better-performing test.