When the global pandemic hit, the U.S. Food and Drug Administration (FDA) authorized several diagnostic tests to be rushed to market ahead of a full official approval process. When the FDA takes such an action through Emergency Use Authorizations (EAUs), the emergency-use product remains in regulatory review to be cleared or rejected upon completion of the process. However, until that time, no public records are available as to the product’s regulatory application, efficacy, or safety profile.
But data does become available – the government begins tracking performance problems reported by hospitals, clinicians, and patients themselves, using the Medical Device Report (MDR) system for capturing adverse events (AEs). Basil Systems has a powerful system for researching such preliminary government health data, and it tells an interesting story.
Quality and performance insights revealed
The FDA uses product codes to organize related products. Of the product codes that cover COVID tests that received EUAs, three (QKO, QKP, and QJR) cover almost all reported AEs through August 2021.
Three-fourths of all reported COVID-19 test AEs mention false positive results. This is a problem, but can be remedied by another test. More importantly, a quarter mention false negatives, which suggests thousands of infected Americans would not have known to quarantine themselves and could be out spreading the virus.
Oddly, with hundreds of millions of COVID-19 tests being performed in the U.S. (Data from U.S. Centers for Disease Control and Prevention), the trend of reported AEs is increasing but still remains suspiciously low. This is consistent with the commonly held belief that the vast majority of AEs do not get reported.
All manufacturers have relatively low reported AE rates, although the number of tests performed by manufacturer is unknown. Curiously, Roche and BD tend to have many AEs that mention false results in the narrative, but are not coded as “false negative” or “false positive” device problems.
Despite these low AE rates and short time on market, many manufacturers have already had several recalls on their EUA-cleared COVID-19 tests. This indicates the AEs may be serious and warrant an urgent change to the test, or the manufacturer has realized there is a problem with the test beyond what the reported AE data indicates.
Questions for the industry to resolve
Again, this data does not tell the whole story, but it does shed important light on what’s happening – and what needs to happen – to fully understand the efficacy of these tests. For example:
- Are the false result rates really this low? In the mad rush to administer these tests, we suspect reporting rates are low and the true AE rate is likely higher than this data shows.
- Why do so many reported AEs mention false results in the narrative text but are not coded as false result device problems?
- With the sheer volume of tests administered to date, what is the actual rate of false test results (either positive or negative), especially given the impact of “false negative” results on containing the spread of COVID-19?
Basil Systems’ unique ability to analyze performance data highlights how critical it is assess the accuracy of AE coding versus the details reported in the narrative. Parsing the narrative text may offer significantly different results.
With Basil Systems’ new ability to quickly assess AE problems, codes, and rates, it highlights how critical it is for industry stakeholders and regulators to assess the accuracy of AE coding vs. the details reported in the narrative. Parsing the narrative text may offer significantly different results than just looking at the device problem or patient problem codes.
It also is essential to characterize the nature and severity of the AEs in order to better predict the odds of a recall. Clearly, recalls are not based on the quantity of adverse events alone. With several major recalls in the past few weeks of both physician-based and home-administered COVID-19 tests, these AEs suggest real problems – and manufacturers and the FDA need to move quickly to address them.
Navigate healthcare data with precision.
Basil leverages AI/ML to provide leading healthcare enterprises with unprecedented analytics and insights that guide effective product and market strategies.