Validation and Performance Assessment — MCQs

10 questions

Specificity of a diagnostic test is defined as:

Which of the following attributes are essential for an ideal screening test?

For evaluating the functioning of a health center, which is the most important determinant for assessing clinical management?

Which of the following statements accurately describes the relationship between quality assurance (QA), quality control (QC), internal quality assurance (IQA), and external quality assurance (EQA)?

All of the following are done to remove Confounding except:

Which measure indicates the diagnostic power of a test to correctly identify those with a disease?

STEPwise approach to surveillance for Non-Communicable diseases step 2 is

Triple assessment for carcinoma breast includes:

A research team develops an AI algorithm using 100,000 CT scans from multiple institutions. The algorithm shows excellent performance (AUC 0.96) but requires extensive computational resources. To deploy it in resource-limited settings, they propose model compression techniques. Evaluate the potential trade-offs and propose the most balanced approach.

Q10

A radiology department is evaluating two AI algorithms for fracture detection. Algorithm A has AUC-ROC of 0.95, while Algorithm B has AUC-ROC of 0.92 but provides explainable results showing which image regions influenced its decision. Considering clinical implementation and medicolegal aspects, which statement best evaluates the choice?

Validation and Performance Assessment Indian Medical PG Practice Questions and MCQs

Question 1: Specificity of a diagnostic test is defined as:

A. 0.95 (Correct Answer)
B. 0.05
C. 0.4
D. 0.8

Explanation: ***0.95*** - **Specificity** is the proportion of individuals without disease who test negative, calculated as **TN/(TN+FP)**. - A specificity of 0.95 (95%) indicates an excellent test that correctly identifies 95% of healthy individuals as negative. *0.05* - This value represents the **false positive rate** (1 - specificity), not specificity itself. - A specificity of 0.05 would mean only 5% of healthy individuals test negative, indicating a very poor test. *0.4* - This value is too low for specificity and could represent other test parameters like **positive predictive value**. - A specificity of 0.4 would incorrectly classify 60% of healthy individuals as positive, making the test clinically unreliable. *0.8* - This value typically represents **sensitivity**, which is the proportion of diseased individuals who test positive. - **Sensitivity** is calculated as **TP/(TP+FN)**, which is different from specificity that focuses on healthy individuals.

Question 2: Which of the following attributes are essential for an ideal screening test?

A. Safe
B. Reliable
C. Valid
D. All of the options (Correct Answer)

Explanation: ***All of the options*** - An ideal screening test must possess **all three essential attributes**: safety, reliability, and validity. - **Safe**: Minimizes harm to participants and ensures ethical implementation - **Reliable**: Produces consistent, reproducible results with minimal random error - **Valid**: Accurately measures what it intends to measure (high sensitivity and specificity) - These three attributes work together as fundamental requirements for any effective screening program, ensuring that early detection benefits outweigh potential risks. *Safe (alone)* - While safety is absolutely essential, it is **not sufficient by itself** to make an ideal screening test. - A test that is safe but unreliable or invalid would produce inconsistent or inaccurate results, rendering it ineffective for screening purposes. *Reliable (alone)* - Reliability ensures consistent results, which is crucial, but **reliability alone is insufficient**. - A test can be highly reliable (consistently giving the same result) yet completely invalid if it measures the wrong thing or is unsafe. *Valid (alone)* - Validity is critical for accurate measurement, but **validity alone does not make a test ideal**. - Even a valid test must be safe to protect participants and reliable to ensure consistency across different settings and times.

Question 3: For evaluating the functioning of a health center, which is the most important determinant for assessing clinical management?

A. Structure
B. Input
C. Process (Correct Answer)
D. Outcome
E. Output

Explanation: ***Process*** - Evaluating the **process** involves assessing the actual delivery of care, including adherence to clinical guidelines, patient-provider interactions, and the timeliness and appropriateness of services. This directly reflects the quality of **clinical management**. - It focuses on *how* care is provided, which is crucial for identifying areas of strength and weakness in the day-to-day operations of a health center's clinical functions. *Structure* - **Structure** refers to the resources and settings in which care is provided, such as facilities, equipment, staff qualifications, and organizational policies. - While important, a good structure does not guarantee good clinical management; the structure offers the potential for quality, but the actual delivery of care (process) is what matters most for assessment. *Input* - **Input** is a broad term often overlapping with structure, referring to the resources poured into the system like funding, staff, and materials. - Like structure, input provides the necessary components, but evaluating them alone does not directly assess the *effectiveness* or *quality* of clinical management. *Output* - **Output** refers to the immediate results of service delivery, such as the number of patients seen, procedures performed, or services rendered. - While outputs can be measured, they represent quantity rather than quality and do not directly assess the appropriateness or effectiveness of clinical management itself. *Outcome* - **Outcome** measures the end results of care, such as patient health status, satisfaction, or mortality rates. - While outcomes are critical, they are often influenced by many factors beyond direct clinical management (e.g., patient adherence, social determinants of health) and may not immediately reflect the quality of the *process* of care delivery itself.

Question 4: Which of the following statements accurately describes the relationship between quality assurance (QA), quality control (QC), internal quality assurance (IQA), and external quality assurance (EQA)?

A. Quality Control (QC) is a process that supports Quality Assurance (QA).
B. Quality Control (QC) and Quality Assurance (QA) are distinct but interrelated processes.
C. Quality Assurance (QA) focuses solely on compliance and excludes Quality Control (QC).
D. Quality Assurance (QA) includes Quality Control (QC), Internal Quality Assurance (IQA), and External Quality Assurance (EQA). (Correct Answer)

Explanation: ***Quality Assurance (QA) includes Quality Control (QC), Internal Quality Assurance (IQA), and External Quality Assurance (EQA).*** - **Quality Assurance (QA)** is the comprehensive, overarching system that encompasses all systematic activities designed to ensure quality throughout the entire process—from planning and design to implementation and evaluation. - **Quality Control (QC)** is an integral component within QA that focuses on operational techniques and activities used to fulfill quality requirements and detect defects in the final product or service. - **Internal Quality Assurance (IQA)** refers to quality assessment activities conducted within the organization itself (self-assessment, internal audits). - **External Quality Assurance (EQA)** involves quality assessment by external agencies (proficiency testing, external audits, accreditation). - All three (QC, IQA, EQA) function as **components within the broader QA framework**, making this the most comprehensive and accurate description of their relationship. *Quality Control (QC) is a process that supports Quality Assurance (QA).* - While this statement is true, it is incomplete and understates the relationship. - QC is not merely "supportive" but is an **integral operational component** embedded within the QA system. - This option fails to capture the comprehensive hierarchical relationship where QA serves as the umbrella framework encompassing QC, IQA, and EQA. *Quality Control (QC) and Quality Assurance (QA) are distinct but interrelated processes.* - From an operational perspective, QA (proactive, prevention-focused) and QC (reactive, detection-focused) do have distinct roles. - However, in quality management frameworks, QC is best understood as a **functional component within the broader QA system** rather than as a separate parallel process. - This option is less precise than the correct answer, which explicitly describes the inclusive hierarchical relationship. *Quality Assurance (QA) focuses solely on compliance and excludes Quality Control (QC).* - This statement is factually incorrect on both counts. - **QA is not limited to compliance**; it encompasses proactive planning, continuous improvement, systematic monitoring, and excellence in all processes—far beyond mere regulatory compliance. - **QA explicitly includes QC** as a core operational function for monitoring and verifying the quality of outputs, making the claim of exclusion completely wrong.

Question 5: All of the following are done to remove Confounding except:

A. Randomization
B. Random Selection
C. Matching
D. Blinding (Correct Answer)

Explanation: ***Blinding*** - **Blinding** is a technique used to minimize **bias** (e.g., observer bias, participant bias) by preventing study participants, researchers, or data analysts from knowing treatment assignments. It does **not address confounding variables**. - Blinding ensures that knowledge of the intervention does not influence the outcomes or their assessment, thereby controlling **information bias** and **performance bias**. *Randomization* - **Randomization** is a key method in experimental studies to control for **confounding variables** by distributing them equally among study groups. - It ensures that each participant has an **equal chance** of being assigned to any study group, thus minimizing systematic differences between groups at baseline. *Random Selection* - **Random selection** (or random sampling) is used primarily to create a sample that is **representative** of the larger population, thereby improving **external validity** and **generalizability**. - While it reduces **selection bias**, random selection is **not a standard method for controlling confounding**. Some sources suggest it may help balance unknown confounders compared to convenience sampling, but it is not classified among the primary confounding control methods (randomization, restriction, matching, stratification, multivariable analysis). - In this question context, if we consider standard epidemiological methods, random selection's role in confounding control is indirect at best. *Matching* - **Matching** is a technique used in observational studies to control for known **confounding variables** by selecting controls who are similar to cases with respect to these confounders. - For example, if age and sex are confounders, controls are matched to cases based on these characteristics to ensure comparability and reduce confounding.

Question 6: Which measure indicates the diagnostic power of a test to correctly identify those with a disease?

A. Negative predictive value
B. Specificity
C. Sensitivity (Correct Answer)
D. Positive predictive value

Explanation: ***Positive predictive value*** - It refers to the probability that subjects with a positive test result truly have the disease, highlighting the test's **diagnostic accuracy** [1]. - A high positive predictive value indicates that the test is effective at diagnosing the disease in the population tested. *Sensitivity* - Sensitivity measures the ability of a test to correctly identify those with the disease (true positives), but does not account for the test result's predictive capability [1]. - It is important for screening, but **not directly the diagnostic power** for those already tested. *Negative predictive value* - This indicates the probability that subjects with a negative test result truly do not have the disease, focusing on true negatives rather than correct diagnosis of the condition [1]. - While informative, it does not assess the ability to correctly diagnose the disease when the result is positive. *Specificity* - Specificity is the measure of a test's ability to correctly identify those without the disease (true negatives), not diagnosing the disease accurately among those tested [1]. - It is essential for determining false positives but not for assessing the overall diagnostic power of a test. **References:** [1] Cross SS. Underwood's Pathology: A Clinical Approach. 6th ed. (Basic Pathology) introduces the student to key general principles of pathology, both as a medical science and as a clinical activity with a vital role in patient care. Part 2 (Disease Mechanisms) provides fundamental knowledge about the cellular and molecular processes involved in diseases, providing the rationale for their treatment. Part 3 (Systematic Pathology) deals in detail with specific diseases, with emphasis on the clinically important aspects., pp. 253-254.

Question 7: STEPwise approach to surveillance for Non-Communicable diseases step 2 is

A. Biochemical Measurement
B. Behavioral measurement
C. Physical measurement (Correct Answer)
D. Emotional Assessment

Explanation: ***Physical measurement*** - The **STEPwise approach** to NCD surveillance involves three steps, with Step 2 specifically focusing on **physical measurements**. - This step includes measurements like **blood pressure**, BMI, weight, height, and waist circumference, which provide crucial data on NCD risk factors. *Biochemical Measurement* - This is typically **Step 3** in the WHO STEPwise approach, focusing on biological measurements from blood or urine samples. - Examples include **blood glucose**, cholesterol levels, and other biomarkers. *Behavioral measurement* - This corresponds to **Step 1** of the WHO STEPwise approach, which involves self-reported data on lifestyle factors. - It covers aspects like **diet**, physical activity, and tobacco/alcohol consumption. *Emotional Assessment* - While emotional and mental health are relevant to overall well-being, **emotional assessment** is not a standard, distinct step in the core WHO STEPwise approach for NCD surveillance. - The STEPs focus on behavioral, physical, and biochemical indicators of NCD risk.

Question 8: Triple assessment for carcinoma breast includes:

A. Observation, Ultrasonography, biopsy/cytology
B. History, clinical examination, biopsy/cytology
C. History, clinical examination, Ultrasonography
D. Clinical examination, Mammography, biopsy/cytology (Correct Answer)

Explanation: ***Clinical examination, Mammography, biopsy/cytology*** - The **triple assessment** for breast carcinoma is a gold standard diagnostic approach comprising **clinical evaluation**, **imaging studies**, and **histopathological assessment**. - **Clinical examination** assesses physical signs, **mammography** provides imaging, and **biopsy/cytology** offers definitive tissue diagnosis. *Observation, Ultrasonography, biopsy/cytology* - **Observation** is not a formal component of the triple assessment; it lacks the specific diagnostic purpose of clinical examination. - While **ultrasonography** is an important imaging modality, particularly for younger women or dense breasts, **mammography** is typically the primary imaging component for initial screening in the triple assessment. *History, clinical examination, biopsy/cytology* - **History** is crucial for understanding risk factors and symptom presentation but is considered part of the broader clinical workup rather than one of the specific "triple" components. - This option omits crucial **imaging**, which is a mandatory part of the triple assessment. *History, clinical examination, Ultrasonography* - While history and clinical examination are vital, this option completely lacks a **histopathological component (biopsy/cytology)**, which is essential for definitive diagnosis of malignancy. - This option also specifies **ultrasonography** over mammography, which, while useful, may not be the primary initial imaging component in all triple assessments.

Question 9: A research team develops an AI algorithm using 100,000 CT scans from multiple institutions. The algorithm shows excellent performance (AUC 0.96) but requires extensive computational resources. To deploy it in resource-limited settings, they propose model compression techniques. Evaluate the potential trade-offs and propose the most balanced approach.

A. Model compression always maintains performance while reducing size
B. Use knowledge distillation to train a smaller model that mimics the larger model while accepting minimal performance decrease (Correct Answer)
C. Avoid compression as any performance loss is unacceptable in medical AI
D. Random pruning of neural network connections is sufficient

Explanation: ***Use knowledge distillation to train a smaller model that mimics the larger model while accepting minimal performance decrease*** - **Knowledge distillation** allows a "student" model to learn the complex features of a "teacher" model, significantly reducing **computational footprint** while preserving high **diagnostic accuracy**. - This approach is the most balanced for **resource-limited settings**, as it optimizes the trade-off between **model size** and the high **AUC** required for clinical safety. *Model compression always maintains performance while reducing size* - This is incorrect because compression techniques like **quantization** or **pruning** often result in some degree of **information loss** or degradation in metric sensitivity. - The goal of compression is to minimize this loss, but it is not a guaranteed consequence of the process. *Avoid compression as any performance loss is unacceptable in medical AI* - While accuracy is critical, failing to compress the model makes it unusable in **edge devices** or areas with low **processing power**, hindering medical access. - Medical AI deployment requires a pragmatic balance between **idealistic performance** and **practical utility** in real-world clinical environments. *Random pruning of neural network connections is sufficient* - **Random pruning** is suboptimal and lacks the strategic precision needed to maintain the **AUC 0.96** performance level required for radiology. - Effective model optimization requires **structured pruning** or **weight-based selection** to ensure critical diagnostic features are not inadvertently deleted.

Question 10: A radiology department is evaluating two AI algorithms for fracture detection. Algorithm A has AUC-ROC of 0.95, while Algorithm B has AUC-ROC of 0.92 but provides explainable results showing which image regions influenced its decision. Considering clinical implementation and medicolegal aspects, which statement best evaluates the choice?

A. Algorithm A should always be chosen due to superior performance metrics
B. Algorithm B may be preferred despite lower AUC due to interpretability and accountability (Correct Answer)
C. AUC-ROC is the only relevant metric for clinical decision making
D. The difference in AUC is clinically insignificant so both are equivalent

Explanation: ***Algorithm B may be preferred despite lower AUC due to interpretability and accountability*** - **Explainable AI (XAI)** is critical in medicine because it allows clinicians to verify the **reasoning process**, ensuring the algorithm isn't relying on irrelevant artifacts. - High **interpretability** facilitates **medicolegal accountability** and builds trust, which are often prioritized over marginal gains in statistical performance metrics like **AUC-ROC**. *Algorithm A should always be chosen due to superior performance metrics* - Relying solely on **performance metrics** ignores the "black box" problem, where a model may have high accuracy but fail unexpectedly in **real-world clinical scenarios**. - Without **spatial localization** or explanation, clinicians cannot easily distinguish between a true positive and a **spurious correlation** detected by the AI. *AUC-ROC is the only relevant metric for clinical decision making* - **AUC-ROC** measures general discriminatory power but does not account for **clinical utility**, workflow integration, or the safety implications of **false negatives**. - Other metrics such as **Positive Predictive Value (PPV)** and **Explainability** are equally vital for determining if a tool is safe and effective for bedside use. *The difference in AUC is clinically insignificant so both are equivalent* - A difference between **0.95 and 0.92** can be statistically and clinically significant depending on the **prevalence** of the condition and the volume of images processed. - Labeling them as **equivalent** overlooks the qualitative advantage of **explainability**, which fundamentally changes how the radiologist interacts with the software.

Want unlimited practice?

Get full access to all questions, explanations, and performance tracking.

Start For Free