ROC curve analysis

Sensitivity & Specificity - Test True Tellers

• Sensitivity (True Positive Rate): Probability of a positive test in a patient with the disease.

  • Formula: $TP / (TP + FN)$
  • 📌 SNOUT: a highly SeNsitive test, when Negative, rules OUT the disease.
  • Ideal for screening tests.

• Specificity (True Negative Rate): Probability of a negative test in a patient without the disease.

  • Formula: $TN / (TN + FP)$
  • 📌 SPIN: a highly SPecific test, when Positive, rules IN the disease.
  • Ideal for confirmatory tests.

ROC Curve Analysis

• Plots Sensitivity (y-axis) vs. 1-Specificity (x-axis) at all cut-off points.
• Area Under the Curve (AUC) reflects test accuracy:
  • AUC = 1.0: Perfect test.
  • AUC = 0.5: Useless (random chance).

⭐ The optimal cut-off point on an ROC curve is typically the one closest to the top-left corner, maximizing both sensitivity and specificity.

ROC Curve Construction - Dot-to-Dot Diagnosis

curve showing sensitivity vs. 1-specificity)

• Plotting the Curve: An ROC curve graphs a test's performance across all possible cut-off points.
  • Y-axis: Sensitivity (True Positive Rate, TPR)
    • $Sensitivity = TP / (TP + FN)$
  • X-axis: 1 - Specificity (False Positive Rate, FPR)
    • $1 - Specificity = FP / (TN + FP)$
• Interpretation: Each point on the curve represents a sensitivity/specificity pair for a specific diagnostic threshold.

⭐ The ideal cut-off point is often the one closest to the top-left corner (100% sensitivity, 100% specificity). The overall test accuracy is measured by the Area Under the Curve (AUC). An AUC of 0.5 is random chance; 1.0 is a perfect test.

AUC & Interpretation - The Curve's Clues

• Area Under the Curve (AUC): Represents the overall diagnostic accuracy of a test. It's the probability that the test will correctly rank a random positive patient higher than a random negative one.
• AUC Scale: Ranges from 0.5 (no discrimination) to 1.0 (perfect discrimination).

Interpreting AUC Values:

• AUC = 1.0: Perfect test.
• AUC > 0.9: Excellent.
• AUC 0.8-0.9: Good.
• AUC 0.7-0.8: Fair.
• AUC < 0.7: Poor/Fail.
• AUC = 0.5: No better than chance (the diagonal line).

• Test Comparison: The test with the ↑ AUC is generally the more accurate diagnostic tool.

⭐ Key Pearl: The AUC is a measure of a test's intrinsic accuracy and is independent of the prevalence of the disease in the population being tested.

High‑Yield Points - ⚡ Biggest Takeaways

• ROC curves plot Sensitivity (TPR) vs. 1-Specificity (FPR) across multiple cut-off thresholds.
• The Area Under the Curve (AUC) is a measure of the overall accuracy of a diagnostic test.
• An AUC of 1.0 represents a perfect test; an AUC of 0.5 represents a test with no discriminatory capacity (i.e., chance).
• The optimal cut-off point on the curve is typically closest to the top-left corner (high sensitivity and specificity).
• When comparing two diagnostic tests, the test with the higher AUC is generally more accurate.