Making Real-World Evidence Work for Medicare: What CMS’ Draft Guidance Gets Right and What Needs Work

Credit: Vitalii Vodolazskyi | stock.adobe.com

CMS’ draft guidance on study protocols using real-world data is a welcome step toward greater rigor, transparency, and consistency in Medicare coverage decisions. This guidance is centered on HARPER+, an adaptation of the HARmonized Protocol Template to Enhance Reproducibility (HARPER). Although HARPER+ provides a valuable structure for researchers developing study protocols for CMS submissions, several refinements could improve its practical value and cross-agency alignment.

1. Clarify HARPER+ Differences to Enable Harmonization

There have been growing calls for harmonization of real world evidence (RWE) guidance across agencies and jurisdictions, most recently highlighted in work by Claire et al. through the IDERHA initiative and by Sarri and Hernandez’s 2024 analysis of global RWE frameworks. CMS’ draft guidance presents an opportunity to advance this goal by more clearly delineating how HARPER+ differs from the widely used HARPER template. One practical approach would be to include HARPER+ modifications in an appendix, enabling researchers to adapt a single protocol for multiple agency submissions with minimal rework.

2. Replace Ambiguous Terminology

The terms “retrospective” and “prospective” are often used inconsistently across research settings. Because these terms can reference different aspects of study timing relative to data collection or outcome measurement, they may create confusion. Replacing them with “secondary” (existing data sources) and “primary” (de novo collection) would provide clearer guidance for researchers and better reflect modern RWE practices.

3. Expand Protocol Registration Options

While CMS rightly emphasizes transparency through study registration, limiting protocol submission to ClinicalTrials.gov may unnecessarily constrain researchers. I recommend acknowledging additional recognized sites, such as the ISPOR-ISPE Real World Evidence Study Registry and the HMA-EMA Catalogue of Real-world Data Studies, both widely used internationally. This would offer flexibility while maintaining accountability.

4. Address International Data Transferability

As the guidance appropriately notes, generalizability is a critical consideration when evaluating real-world studies for healthcare decision-making. However, the guidance stops short of discussing studies conducted outside of the United States. To facilitate the use of robust studies conducted outside the CMS-eligible population (including international studies), the guidance should note the importance of demonstrating the transferability of the data or the transportability of the study results so that CMS reviewers have the proper context when evaluating such studies.

5. Reference Emerging Methodological Frameworks

HARPER references several foundational frameworks and tools to support the documentation and justification of study design and data source decisions. While this provides researchers with a strong starting point, the HARPER+ framework represents a more recent evolution. Incorporating references to emerging tools, such as the Structured Process to Identify Fit-for-Purpose Study Design and Data to Generate Valid and Transparent Real-World Evidence for Regulatory Uses (SPIFD2) and Staging and Clean Room Constructs, would offer additional resources to promote best practices and enhance the scientific credibility of real-world studies submitted for Medicare coverage decisions.

Looking Ahead

CMS’s draft guidance represents meaningful progress toward aligning RWE best practices with Medicare coverage requirements. Continued collaboration with the broader research community will be essential as the agency finalizes its recommendations to ensure practical, scientifically rigorous guidance. A strong framework will help ensure that Medicare coverage decisions are supported by high-quality evidence, ultimately advancing beneficial outcomes.

About the Author

Patrick (Pat) Arena, PhD, MPH is a Director of Science at Aetion, Inc. with more than five years of industry experience. Pat joined Aetion in 2022 and currently focuses on applying epidemiologic expertise to original RWD research projects, understanding the role of RWE in regulatory/HTA decisions, and providing regulatory consulting support for various clients. Prior to Aetion, Pat worked at Pfizer, Inc., focusing on post-marketing safety studies as well as Phase II/III vaccine clinical trials. He also has applied global health experience contributing to research projects in both Southeast Asia and Central Africa. Throughout his career, Pat has authored various publications using RWD in multiple jurisdictions (e.g., the US, China, Africa, and Europe) and has led initiatives exploring the role of RWD/E transferability and transportability. Pat received his BS from Boston College, MPH from Columbia University, and PhD in Epidemiology from UCLA; his doctoral training focused on maternal immunization safety, COVID-19, and pharmacovigilance.