A new study into how well real-world evidence could replicate clinical trials published over the course of a year highlights some of the limitations that still exist.
The study, published in the Journal of the American Medical Association, found that only 15 percent of clinical trials included in the analysis could be feasibly replicated using currently available real-world data sources. Despite opportunities to supplement or even replace some trials, “observational methods are not likely to obviate the need for traditional clinical trials,” the researchers wrote. At the same time, they wrote that the data suggest that RWE could complement clinical trials, through examining concordance between randomized and observational studies and comparing the generalizability of a trial population with the real-world population.
RWE comes from analysis of data from sources like electronic health records, patient-reported outcome measures, insurance claims, medical devices and mobile devices. Under the 21st Century Cures Act of 2016, the Food and Drug Administration has prioritized research into RWE and how it can support regulatory decision making.
To conduct the study, researchers at the Mayo Clinic, Yale University, University of California San Francisco and Columbia University searched PubMed to compile a sample of 220 trials – whether or not they were randomized – published across the seven highest-impact medical journals in 2017. They found that only 33 of the studies could be replicated using observational data. That was because for the most part, things like primary endpoints, inclusion and exclusion criteria, indication and intervention – all crucial factors in any clinical trial – could not be routinely ascertained from real-world data sources.
“I think there are lots of ways that RWE will be used for medical product evaluation, but for these data to be a suitable replacement, work is still needed to enhance the data that are available,” one of the researchers, Yale professor of medicine Dr. Joseph Ross, wrote in an email. “Working with current real-world data sources, I believe large, safety-oriented evaluations are useful and feasible. But I think more needs to be done to better discern product effectiveness using real-world data.”
Ross noted that his team is doing a lot of work with the FDA and National Evaluation System for health Technology Coordinating Center to test whether aggregating multiple real-world data sources – including EHR and pharmacy data, PROMs and data from digital devices – can be used to support pragmatic clinical trials.
Photo: oonal, Getty Images
