11/25/2011

1
Diagnostic Test Statistics
Phil Hahn
December 3, 2011
Copyright 03/12/2011 by Phil Hahn
Diagnostic Test Statistics
Predictive values depend on prevalence of disease.
The lower the prevalence, the lower the predictive
value of a positive test.
A good diagnostic test may not be appropriate for
screening a general population with low prevalence of
disease.
Diagnostic Test Statistics
Likelihood ratios are not dependent on prevalence.
Positive Likelihood ratio indicates how much the odds
f di i d if th t t lt i iti of disease are increased if the test result is positive.
Diagnostic Test
Test
Disease
Test
Disease-free
Richard K. Riegleman.
Chapter 15: Diagnostic Discrimination of Tests
In: Studying a Study and Testing a Test:
How to read the medical literature.
Little, Brown and Company, Boston 1981
Gold Standard
The criteria use to unequivocally define
the presence of a disease
Considered the truth
the best we got
Sensitivity
Proportion (%)
Diseased (gold standard) Diseased (gold standard)
who test positive (new test)
% of true positives
11/25/2011
2
Specificity
Proportion (%)
Disease free (gold standard) Disease-free (gold standard)
who test negative (new test)
% of true negatives
2x2 Table
Test
Gold Standard
Diseased Disease-free
Positive True Positives False Positives Positive True Positives
A
False Positives
B
Negative False Negatives
C
True Negatives
D
Sensitivity (% of true positives) = A / A+C
Specificity (% of true negatives) = D / B+D
50% Prevalence
Test
Gold Standard
500
Diseased
500
Disease-free
Positive 400
A
50
B
Negative 100
C
450
D
Sensitivity (% of true positives) = A / A+C = 400/500 = 80%
Specificity (% of true negatives) = D / B+D = 450/500 = 90%
10% Prevalence
Test
Gold Standard
100
Diseased
900
Disease-free
Positive 80
True Positives
90
False Positives
Negative 20 810
Sensitivity (% of true positives) = A / A+C = 80/100 = 80%
Specificity (% of true negatives) = D / B+D = 90/900 = 90%
1% Prevalence
Test
Gold Standard
100
Diseased
9,900
Disease-free
Positive 198
False Positives
Negative 9,702
False Positives > Total number of diseased
The actual number of
false positives and negatives
is dependent on the is dependent on the
prevalence of the disease.
11/25/2011
3
Predictive Values
of
Positive & Negative Tests
Clinical Questions
Positive Test
Likelihood individual diseased?
Negative Test
Likelihood individual disease-free?
Predictive Values
The probability of the disease
being present or absent after
obtaining the results of the test. obtaining the results of the test.
Predictive Values
Test
Gold Standard
Totals
Diseased Disease-free
Positive True False Total
Positives
A
Positives
B
Positives
A+B
Negative False
Negatives
C
True
Negatives
D
Total
Negatives
C+D
Positive Predictive Value (PPV) = A / A+B
Negative Predictive Value (NPV) = D / C+D
50% Prevalence
Test
Gold Standard
Totals
500
Diseased
500
Disease-free
Positive 400
A
50
B
450
A+B
Negative 100
C
450
D
550
C+D
Positive Predictive Value (PPV) = A / A+B = 400/450 = 89%
Negative Predictive Value (NPV) = D / C+D = 450/550 = 82%
10% Prevalence
Test
Gold Standard
Totals
100
Diseased
900
Disease-free
Positive 80
A
90
B
170
A+B
Negative 20
C
810
D
830
C+D
Positive Predictive Value (PPV) = A / A+B = 80/170 = 47%
Negative Predictive Value (NPV) = D / C+D = 810/830 = 98%
11/25/2011
4
Important Messages
Predictive values depend on prevalence
The lower the prevalence, the lower The lower the prevalence, the lower
the predictive value of a positive test
A test may be useful for diagnosis in a
population with high suspicion of disease,
but not appropriate for screening a general
population (with low prevalence)
Predictive Values versus Prevalence
Sensitivity & Specificity set at 95%
Prevalence
David Sackett et al. (1991)
Clinical Epidemiology 2
nd
edition, p. 90.
Prevalence is Key
Example
Testing for AIDS
High risk groups High-risk groups
Homosexuals: sexual contact
Street drug users: sharing contaminated
needles
David Steiner, Geoffrey Norman
PDQ Epidiemology 2
nd
Edition (1998)
HIV Tests
Gold Standard
Western blot test
Virtually perfect Virtually perfect
Less expensive ELISA test
50% Prevalence
(homosexuals in New York City)
Test
(ELISA)
Gold Standard (Western blot)
508
Diseased
492
Disease-free
Positive 498
A
4
B
Negative 10
C
488
D
Sensitivity= A / A+C = 498/508 = 98%
Specificity = D / B+D = 488/492 = 99%
50% Prevalence
(homosexuals in New York City)
Test
(ELISA)
Gold Standard (Western blot)
Totals
1000
508
Diseased
492
Disease-free
Positive 498
A
4
B
502
A+B
Negative 10
C
488
D
498
C+D
Positive Predictive Value (PPV) = A / A+B = 498/502 = 99%
Negative Predictive Value (NPV) = D / C+D = 488/498 = 98%
11/25/2011
5
0.2% Prevalence
(general population of men in New York City)
Test
(ELISA)
Gold Standard (Western blot)
Totals
1,000,000
2000
Diseased
998,000
Disease-free
Positive 1960
A
7984
B
9944
A+B
Negative 40
C
990,016
D
990,056
C+D
Positive Predictive Value (PPV) = A / A+B = 1960/9944 = 20%
Negative Predictive Value (NPV) = D / C+D = 990,016/990,056 = 99%
0.2% Prevalence
(general population of New York City)
Test
(ELISA)
Gold Standard (Western blot)
Totals
1,000,000
2000
Diseased
998,000
Disease-free
Positive 1960 7984 9944
Negative 40 990,016 990,056
Positive Predictive Value (PPV) = 20%
A positive test if nearly uninterpretable, 80% of tests false positives
Create massive anxiety among those testing positive
Why mandatory HIV testing (screening) cannot work!
Refining Clinical Diagnosis
with Likelihood Ratios
The Lancet Handbook of Essential Concepts in
Clinical Research (2006)
Kenneth Schulz and David Grimes
Likelihood Ratio (LR)
% of ill people with a given test result
% of well people with the same test result
Likelihood of test result in person with disease
Likelihood of test result in person without disease
Example 2x2 table
Test
Disease
Present
(n=15)
Absent
(n=85)
Positive 12
(80%)
5
(6%)
Negative 3
(20%)
80
(94%)
15 people are sick and 12 (80%) have a true-positive test
(sensitivity)
85 people are well but five (6%) have a false-positive test
LR+ (positive test)
Test
Disease
Present
(n=15)
Absent
(n=85)
Positive 12
(80%)
5
(6%)
Negative 3
(20%)
80
(94%)
LR+ = true-positive rate / false-positive rate = 80%/6% = 13
People with disease are 13x more likely to have a positive test
than those who are well (without disease).
11/25/2011
6
LR+ (positive test)
Test
Disease
Present
(n=15)
Absent
(n=85)
Positive (80%) (6%)
Negative (20%) (94%)
LR+ = 80%/6% = 13
Indicates how much the odds of disease is increased if test
result is positive (13x).
Attractive features
Likelihood ratios are not dependent on
prevalence
Can be applied to a specific patient Can be applied to a specific patient
Putting LRs to Work
Bayesian principle
Pretest odds of disease x LR
= post test odds of disease p
Unlike gamblers, clinicians dont think
in terms of odds
Nomogram: Convert odds to probabilities
LR+ 13
15%
LR+ 13
71%
15%
LR+ 13
Assume pretest probability is 25%
25%
11/25/2011
7
LR+ 13
81%
Assume pretest probability is 25%
25%
Calculator by Alan Schwartz is Free Software.
College of Medicine at Chicago
71%
15%
LR+ = 14
Change Prevalence to 25%
25%
82%
LR+ = 14
11/25/2011
8
Rule of Thumb
LR = 1.0 not helpful
% of sick and well people with test result
is the same
LR Post-test Probability
1 to 2 Post test change not important
2 to 5 Small increase
5 to 10 Moderate increase
> 10 Large increase
Take Home Points
Predictive values
depend on prevalence
A good diagnostic test may not be appropriate for
i l l ti ( ith l l screening a general population (with low prevalence
of disease)
Likelihood Ratios
Not affected by prevalence
Nomogram: Pretest probability of disease x LR
= post test probability of disease
Can be related to a the patient in your clinic
References
Douglas G Altman, J Martin Bland.
Diagnostic tests. 1: Sensitivity and specificity.
BMJ 1994 J un 11;308(6943):1552.
Altman DG, Bland J M.
Diagnostic tests 2: Predictive values.
BMJ 1994 J ul 9;309(6947):102.
J onathan J Deeks, Altman DG.
Diagnostic tests 4: likelihood ratios.
BMJ 2004 J ul 17;329(7458):168-9.