Real-World Problems for Real-World Data Guidance?

In September, FDA published guidance on strategies for accessing real-world data (RWD) from electronic health records (EHRs) and medical claims in support of submissions for drug and biologic approval. As Pharm Exec’s Jill Wechsler wrote (Oct. 1), “the diversity and limitations of this information has stymied broader adoption for documenting drug effectiveness.” In asking drug developers “to weigh in on strategies utilizing RWD to support regulatory decisions and to help define treatment exposure, outcomes, data quality assurance, and data quality control procedures,” FDA’s move looks toward satisfying the demand for faster development of medical products.

However, for Miruna Sasu, PhD, chief strategy officer of RWD company COTA, the FDA guidance must go further. Sasu welcomes the fact that FDA “is recognizing that RWD needs to be pursued and is addressing how it would accept RWD as part of submission packages.” She is also pleased that the guidance talks about issues relating to regulatory packages that are sent back to companies by FDA—data can be inconsistent, have a high rate of “missingness,” or may not be representative of the holistic population. The problem, Sasu explains, is that the guidance doesn’t specify how companies can overcome these hurdles. And while it acknowledges that data from different sites are not consistent—e.g., those from a community oncology site differ from an academic site—neither does the guidance “clarify what FDA will accept to prove that the data is holistically representative.”

The way that data is entered into EHRs can vary widely. “One doctor may enter the data in millimeters, another may enter the data in words, a third doctor may record something very non-quantitative about disease progression,” explains Sasu. What COTA would like to see from FDA is a mandate around healthcare professionals (HCPs) collaborating to co-source the data. “At the moment, we’re dancing around the data missingness because we’re taking whatever the HCP is putting in. We need FDA to say, ‘You all need to work together on this.’”

The guidance states that artificial intelligence (AI), including natural language processing, machine learning, and deep learning, may result in more rapid processing of unstructured electronic healthcare data, and goes on to identify the use cases for AI, including extracting data elements from unstructured and structured fields in EHRs. “However,” says Sasu, “we have yet to prove that AI can consistently deliver reliable results independently, without the use of medical personnel quality checks, when extracting data elements from EHRs, especially in unstructured fields. Utilizing this type of technology may result in additional inconsistencies until algorithms are trained on very large data-
sets.” She goes on, “By communicating its support for AI in data abstraction in this guidance, will FDA be willing to accept the likely inconsistencies in the data that AI introduces? Or will it provide guidance on how these processes may get better on pulling the data?”

In light of concerns around the September guidance, FDA issued a follow-up document in October to further address the incorporation of RWD/RWE into submissions to the agency. The October guidance states that companies should collect and convert or “map” such data collected outside of clinical trials to fit agency e-standards for submitting information in investigational new drug applications and applications for generic drugs and biotech therapies, Jill Wechsler reported (PharmExec.com, Oct. 28). The guidance adds however that “all parties concede that this will be a lengthy and difficult process.”

Sasu welcomes the further information but adds, “What the agency is saying is that the data in EHRs have to be similar to clinical trial data. But it has not mandated that HCPs collect that information in the way a clinical trial is typically done. So FDA acknowledges that there is a lot of missingness in RWD, but at the same time it wants no missingness, as with clinical trial data.”

Sasu concludes, “We always appreciate better guidance. I’m a drug developer at heart (having held positions in clinical trials and biometrics at Bristol Myers Squibb and Johnson & Johnson); the reason I got into this business is because I want patients to get better access to medicines faster. An FDA mandate on collaboration between HCPs and the companies collecting and curating health data would make things clearer and easier, and improve things for patients down the line.”

Julian Upton is Pharm Exec’s European and Online Editor. He can be reached at jupton@mjhlifesciences.com.