Comparing COVID-19 antibody tests
The performance metrics for the serological tests FDA has authorized so far serve as guideposts for sizing up the next wave of tests to come to market.
FDA has granted Emergency Use Authorization to nine COVID-19 serological tests, and is reviewing at least 140 more. Additional tests could receive EUA via the agency's "umbrella" pathway, in which tests are validated by certain government agencies outside of FDA, such as the serological test validation program led by NIH's National Cancer Institute (NCI).
The tests, which detect antibodies against SARS-CoV-2 in patient blood, are used to determine who has been exposed to the virus. They don’t diagnose infection, but could guide individual and public health decisions such as where to allocate resources, and who can go back to work.
Tests with binary results are typically evaluated based on sensitivity and specificity, which are characteristics intrinsic to the test.
Objectively defining sensitivity and specificity requires having a reference standard -- a test universally recognized as the best available method for establishing the presence or absence of a condition. When no reference standard is available, sensitivity and specificity are defined as positive percent agreement (PPA) and negative percent agreement (NPA) with another test of the developer's choice (see “The Quarantiner’s Guide to Evaluating COVID-19 Tests”).
Figure: The quarantiner’s guide to evaluating COVID-19 tests
Each FDA-authorized serological test has an instructions for use document that reports its sensitivity and specificity in the form of PPA and NPA with a reverse transcription polymerase chain reaction (RT-PCR) test for SARS-CoV-2 infection in the same individuals, along with 95% confidence intervals (CI) for each value. (The 95% CI represents the range of values that the user can be 95% certain contains the test characteristics).
For sensitivity, or PPA, the width of the 95% CI depends on the number of patient samples used to validate the test. For specificity, or NPA, the number of negative control samples are the main influence.
Some manufacturers report PPA and NPA values for samples collected at different time intervals after symptom onset; the data shown represent results for the latest time interval reported.
The shaded area in the diagram below represents the 90% sensitivity and 95% specificity cutoffs required for serological tests to receive EUA via the umbrella pathway. The diagram plots the values of nine tests that have received EUA from FDA, four of which don’t measure up to the umbrella pathway guidelines (see “Serological Test Report Card” and “Characteristics & Validation of COVID-19 Serological Tests with FDA EUA).
Figure: Serology test report card
Table: Characteristics & validation of COVID-19 serological tests with FDA EUA
The clinical relevance of a test depends on the prevalence of the condition it detects.
How much one can trust a positive or negative result in a given clinical population is quantified by a test's positive predictive value (PPV) or negative predictive value (NPV). The PPV and NPV of a test with a fixed sensitivity/PPA and specificity/NPA changes as function of case prevalence in the population.
The rapid advancement of the COVID-19 crisis has limited the availability of key resources and information needed to fully validate the performance of serological tests.
