Optimizing cut-off values

On this page

Sensitivity & Specificity - The Diagnostic Duo

  • Sensitivity (True Positive Rate): Correctly identifies those with the disease.

    • Formula: $TP / (TP + FN)$
    • 📌 SN-N-OUT: Highly Sensitive test, when Negative, rules OUT disease. Ideal for screening.
  • Specificity (True Negative Rate): Correctly identifies those without the disease.

    • Formula: $TN / (TN + FP)$
    • 📌 SP-P-IN: Highly Specific test, when Positive, rules IN disease. Ideal for confirming a diagnosis.
  • Cut-off Point: There's an inverse relationship. Lowering the cut-off ↑ sensitivity, ↓ specificity.

ROC Curve for Predictive Models curve showing trade-off between sensitivity and specificity)

⭐ The Area Under the Curve (AUC) from an ROC plot measures overall test accuracy. An AUC of 0.5 is random chance; 1.0 is a perfect test. The optimal cut-off is the point on the curve closest to the top-left corner.

Cut-off Values - The Balancing Act

  • The choice of a cut-off for a continuous-data diagnostic test creates a trade-off between correctly identifying those with and without a disease.
  • This balance is visualized using a Receiver Operating Characteristic (ROC) curve.
  • Lowering the threshold: Increases sensitivity (more true positives), but decreases specificity (more false positives). Ideal for screening tests where missing a case is costly.
  • Raising the threshold: Increases specificity (more true negatives), but decreases sensitivity (more false negatives). Best for confirmatory tests to avoid over-diagnosis.

⭐ The ideal cut-off value corresponds to the point on the ROC curve in the upper-left corner, which maximizes the Youden's J statistic (Sensitivity + Specificity - 1).

ROC curve showing optimal cut-off point

ROC Curves - Plotting Performance

ROC curve for optimizing cut-off values

  • A graphical plot that illustrates the diagnostic ability of a test by plotting the true positive rate against the false positive rate at various threshold settings.

    • Y-axis: Sensitivity (True Positive Rate, $TPR$)
    • X-axis: 1 - Specificity (False Positive Rate, $FPR$)
  • Area Under the Curve (AUC): Represents the test's overall ability to discriminate between diseased and healthy individuals.

    • AUC = 1.0: Perfect test (curve hugs top-left corner).
    • AUC > 0.8: Good to excellent test.
    • AUC = 0.5: Useless test (follows the diagonal line of chance).
  • Optimal Cut-off: The point on the curve closest to the top-left corner (0,1) that maximizes sensitivity and specificity.

⭐ The AUC represents the probability that a randomly selected diseased individual has a more abnormal test result than a randomly selected non-diseased individual.

Clinical Context - Picking the Point

ROC curve: WBC to predict bacterial infection

The choice of a cut-off value balances sensitivity and specificity, depending on the clinical objective.

  • High Sensitivity (Screening Tests)

    • Goal: To rule-OUT disease (e.g., initial ELISA).
    • Action: Lower the cut-off value to capture all potential cases.
    • Result: ↑ True Positives, but also ↑ False Positives.
    • 📌 Sn-N-Out: a highly Sensitive test, when Negative, rules Out the disease.
  • High Specificity (Confirmatory Tests)

    • Goal: To rule-IN disease (e.g., Western Blot).
    • Action: Raise the cut-off value to be certain of diagnosis.
    • Result: ↑ True Negatives, but also ↑ False Negatives.
    • 📌 Sp-P-In: a highly Specific test, when Positive, rules In the disease.

⭐ The ideal cut-off on an ROC curve is the point closest to the top-left corner (100% sensitivity, 100% specificity). This point maximizes the Youden Index: $J = Sensitivity + Specificity - 1$.

High-Yield Points - ⚡ Biggest Takeaways

  • Lowering a cut-off increases sensitivity but decreases specificity; ideal for screening tests (e.g., SNOUT).
  • Raising a cut-off increases specificity but decreases sensitivity; preferred for confirmatory tests (e.g., SPIN).
  • ROC curves plot Sensitivity (TPR) vs. 1-Specificity (FPR) to visualize this trade-off.
  • The Area Under the Curve (AUC) reflects the overall diagnostic accuracy of the test.
  • An AUC of 1.0 is a perfect test; 0.5 is equivalent to chance.
  • The optimal cut-off is the point on the curve closest to the top-left corner.

Practice Questions: Optimizing cut-off values

Test your understanding with these related questions

A 40-year-old female volunteers for an invasive study to measure her cardiac function. She has no previous cardiovascular history and takes no medications. With the test subject at rest, the following data is collected using blood tests, intravascular probes, and a closed rebreathing circuit: Blood hemoglobin concentration 14 g/dL Arterial oxygen content 0.22 mL O2/mL Arterial oxygen saturation 98% Venous oxygen content 0.17 mL O2/mL Venous oxygen saturation 78% Oxygen consumption 250 mL/min The patient's pulse is 75/min, respiratory rate is 14/ min, and blood pressure is 125/70 mm Hg. What is the cardiac output of this volunteer?

1 of 5

Flashcards: Optimizing cut-off values

1/10

_____ is the probability that when the disease is present, the test is positive

TAP TO REVEAL ANSWER

_____ is the probability that when the disease is present, the test is positive

Sensitivity

browseSpaceflip

Enjoying this lesson?

Get full access to all lessons, practice questions, and more.

Start Your Free Trial
Optimizing cut-off values - Free USMLE High-Yield Review