COVID-19 test validity: How accurate are the available tests?

7th Jan 2021

Let’s have a look at sensitivity, specificity, and predictive values of laboratory tests in the context of COVID-19

 

RT-PCR

Let’s start off with reverse transcription polymerase chain reaction, or RT-PCR. We’ve heard that RT-PCR has great specificity, which means that a positive test result is usually proof of infection. However, the sensitivity is not great, which means that we miss quite a lot of folks who are really infected. And that’s why we usually request two negative PCR tests before we can send our COVID-19 patients home after they were admitted to the hospital.

Because of the risk of false negatives, the Robert Koch institute in Germany recommends retesting high-risk contacts of COVID-19 patients 5 to 7 days after contact if their initial reverse transcription polymerase chain reaction test was negative.

 

Antibody tests

So what about antibody tests

Take the example of a lateral flow point-of-care antibody test with a reported sensitivity of 93.8% and a specificity of 95.6%.1 Let’s assume we have a population of one million people and the prevalence of past COVID-19 infection is 5%. So 50 000 individuals had the disease and 950 000 did not have the disease.

With a sensitivity of 93.8%, we would detect 93.8% of diseased individuals or 46 900 individuals. And with a specificity of 95.6%, we would detect 95.6% of non-diseased or 908 200 individuals. Therefore, 3100 would be falsely classified as non-diseased or false negatives. And 41 800 would be falsely classified as diseased or false positives. 

Chart of true and false positives and negatives for a lateral flow immunoassay with a sensitivity of 93.8% and specificity of 95.6%.

Figure 1. The number of true and false positives and true and false negatives obtained by using a lateral flow immunoassay with a sensitivity of 93.8% and a specificity of 95.6% in a population of one million people with a prevalence of past COVID-19 infection of 5%.

Given the number of false positives, we would greatly overestimate the level of immunity in this population with this test.

So let’s calculate the predictive values. 

The positive predictive value (PPV) is 46 900, the true positives, divided by 88 700, all people who tested positive, times 100 which equals 53%. Not so great, right?

And what’s the negative predictive value (NPV)? Divide 908 200, the true negatives, by 911 300, all the people who tested negative, times 100 which equals 99.7% for the NPV—pretty good!

Positive predictive values and negative predictive values are calculated for a lateral flow immunoassay with a sensitivity of 93.8% and a specificity of 95.6%.

Figure 2. Positive predictive value (PPV) and negative predictive value (NPV) are calculated for a lateral flow immunoassay with a sensitivity of 93.8% and a specificity of 95.6% used in a population of one million people with a prevalence of past COVID-19 infection of 5%.

So, the problem with these tests that don’t have a near-perfect specificity is that when we use them at the population level, they are pretty useless because even in hard-hit regions, the prevalence of the disease is pretty low.

 

Clickable call to action, "Start learning for free", with direct link to sign up for a free Medmastery trial account.

 

The influence of disease prevalence on test validity of antibody tests

But what happens, when you test a group of 100 000 individuals who had typical COVID-related symptoms one month ago when your region was hardest hit? 

Well, the prevalence of past disease in this group will be much higher. Let’s say just based on their symptoms the probability of them having had the disease is 20%. So 20 000 individuals would have had the disease and 80 000 would have not had the disease. 

Our test, being 93.8% sensitive would pick up 18 760 diseased individuals. Due to its specificity of 95.6%, it would correctly classify 76 480 non-diseased individuals. It follows that 1240 individuals would be falsely classified as non-diseased—false negatives—and 3520 would be falsely classified as diseased—false positives.

Positive and negative test results in a population of 100 000 individuals, with 20% prevalence of COVID-19 infection, using a lateral flow immunoassay with 93.8% sensitivity and 95.6% specificity. Chart.

Figure 3. Positive and negative test results in a population of 100 000 individuals, with a 20% prevalence of COVID-19 infection, using a lateral flow immunoassay with 93.8% sensitivity and 95.6% specificity.

So let’s start with the NPV. 76 480 true negatives divided by all 77 720 individuals who tested negative times one hundred gives us an NPV of 98.4%. So still pretty good and not much different from our previous value of 99.7%.

And what’s the positive predictive value? 18 760 true positives divided by 22 280 individuals who tested positive times one hundred gives us a PPV of 84%. This is much higher than the PPV of 53% when the infection prevalence was 5%. 

Positive predictive value and negative predictive value in a population of 100 000 individuals, with 20% prevalence of COVID-19 infection, using a lateral flow immunoassay with 93.8% sensitivity and 95.6% specificity. Chart.

Figure 4. Positive predictive value (PPV) and negative predictive value (NPV) in a population of 100 000 individuals, with a 20% prevalence of COVID-19 infection, using a lateral flow immunoassay with 93.8% sensitivity and 95.6% specificity.

So many of these lateral flow COVID-19 point-of-care antibody tests will not be very useful at a population level if they don’t have near-perfect specificity. But they are useful if used on patients with a high likelihood of past disease.

 

That’s it for now. If you want to improve your understanding of key concepts in medicine, and improve your clinical skills, make sure to register for a free trial account, which will give you access to free videos and downloads. We’ll help you make the right decisions for yourself and your patients.

References

  1. Johns Hopkins University. 2020. Serology-based tests for COVID-19. Johns Hopkins Universityhttps://www.centerforhealthsecurity.org/