Optimizing cut-off values

Sensitivity & Specificity - The Diagnostic Duo

• Sensitivity (True Positive Rate): Correctly identifies those with the disease.

  • Formula: $TP / (TP + FN)$
  • 📌 SN-N-OUT: Highly Sensitive test, when Negative, rules OUT disease. Ideal for screening.

• Specificity (True Negative Rate): Correctly identifies those without the disease.

  • Formula: $TN / (TN + FP)$
  • 📌 SP-P-IN: Highly Specific test, when Positive, rules IN disease. Ideal for confirming a diagnosis.

• Cut-off Point: There's an inverse relationship. Lowering the cut-off ↑ sensitivity, ↓ specificity.

curve showing trade-off between sensitivity and specificity)

⭐ The Area Under the Curve (AUC) from an ROC plot measures overall test accuracy. An AUC of 0.5 is random chance; 1.0 is a perfect test. The optimal cut-off is the point on the curve closest to the top-left corner.

Cut-off Values - The Balancing Act

• The choice of a cut-off for a continuous-data diagnostic test creates a trade-off between correctly identifying those with and without a disease.
• This balance is visualized using a Receiver Operating Characteristic (ROC) curve.

• Lowering the threshold: Increases sensitivity (more true positives), but decreases specificity (more false positives). Ideal for screening tests where missing a case is costly.
• Raising the threshold: Increases specificity (more true negatives), but decreases sensitivity (more false negatives). Best for confirmatory tests to avoid over-diagnosis.

⭐ The ideal cut-off value corresponds to the point on the ROC curve in the upper-left corner, which maximizes the Youden's J statistic (Sensitivity + Specificity - 1).

ROC Curves - Plotting Performance

• A graphical plot that illustrates the diagnostic ability of a test by plotting the true positive rate against the false positive rate at various threshold settings.

  • Y-axis: Sensitivity (True Positive Rate, $TPR$)
  • X-axis: 1 - Specificity (False Positive Rate, $FPR$)

• Area Under the Curve (AUC): Represents the test's overall ability to discriminate between diseased and healthy individuals.

  • AUC = 1.0: Perfect test (curve hugs top-left corner).
  • AUC > 0.8: Good to excellent test.
  • AUC = 0.5: Useless test (follows the diagonal line of chance).

• Optimal Cut-off: The point on the curve closest to the top-left corner (0,1) that maximizes sensitivity and specificity.

⭐ The AUC represents the probability that a randomly selected diseased individual has a more abnormal test result than a randomly selected non-diseased individual.

Clinical Context - Picking the Point

The choice of a cut-off value balances sensitivity and specificity, depending on the clinical objective.

• High Sensitivity (Screening Tests)

  • Goal: To rule-OUT disease (e.g., initial ELISA).
  • Action: Lower the cut-off value to capture all potential cases.
  • Result: ↑ True Positives, but also ↑ False Positives.
  • 📌 Sn-N-Out: a highly Sensitive test, when Negative, rules Out the disease.

• High Specificity (Confirmatory Tests)

  • Goal: To rule-IN disease (e.g., Western Blot).
  • Action: Raise the cut-off value to be certain of diagnosis.
  • Result: ↑ True Negatives, but also ↑ False Negatives.
  • 📌 Sp-P-In: a highly Specific test, when Positive, rules In the disease.

⭐ The ideal cut-off on an ROC curve is the point closest to the top-left corner (100% sensitivity, 100% specificity). This point maximizes the Youden Index: $J = Sensitivity + Specificity - 1$.

High-Yield Points - ⚡ Biggest Takeaways

• Lowering a cut-off increases sensitivity but decreases specificity; ideal for screening tests (e.g., SNOUT).
• Raising a cut-off increases specificity but decreases sensitivity; preferred for confirmatory tests (e.g., SPIN).
• ROC curves plot Sensitivity (TPR) vs. 1-Specificity (FPR) to visualize this trade-off.
• The Area Under the Curve (AUC) reflects the overall diagnostic accuracy of the test.
• An AUC of 1.0 is a perfect test; 0.5 is equivalent to chance.
• The optimal cut-off is the point on the curve closest to the top-left corner.