By Grammati Sarri, Evie Merinopoulou, Vinusha Kalatharan, and Jason Simeone

The Canadian Agency for Drugs and Technologies in Health’s (CADTH) long-expected guidance on real-world evidence (RWE) is now in the public domain; however, it solely provides standardized (core) reporting practices for manufacturers submitting RWE studies to support their drugs applications, and important aspects of the incorporation of RWE in decision-making are missing.

The CADTH Guidance: Key Themes

• The need for literature reviews to justify decisions regarding the RWE study design and methods.
• The application of frameworks such as target trial emulation to transparently justify each component of the study design including considerations for bias.
• Demonstrating assessment of the extent and direction of biases through structured processes such as the bias quantification methods to gain confidence in RWE findings.
• Clear reference to the importance of reporting participant demographics (e.g., age, sex, gender, race or ethnicity, and socioeconomic factors) to ensure health inequalities are considered in decision-making, especially for historically underrepresented patient populations.

The CADTH Guidance: What’s Included

• Setting up aim and research question
  • Use of PICOTS template to guide literature review.
  • Literature reviews to support epidemiology, background considerations, PICOTS components, and evidence gaps justifying the RWE need.
• Study design
  • Literature review to support the design selection.
  • Causal inference frameworks, such as target trial emulation to guide study designs and methods.
  • Use of study diagrams.
  • Use of Harmonized Protocol Template to Enhance Reproducibility (HARPER).
  • Registration of protocol to credible platforms: Real-World Evidence Registry, Clinical Trials.gov (Observational Study Type Specification), and European Union Electronic Register of Post-Authorisation Studies (EU PAS Register).
• Setting and context
  • RWE transportability from non-Canadian context is accepted if important RWE components are transparently reported (relevance, quality, limitations, generalizability) and background information about the health systems and disease epidemiology by country is provided.
• Data specification and sources
  • Consider all available RWE data sources.
  • Detailed recording of data preparation (access, cleaning, quality control, linkage).
  • For registries, using Registry Evaluation and Quality Standards Tool (REQueST).
  • Consider the use of core outcomes set (COS) for standardized outcome reporting and specific considerations for the use of surrogate outcomes.
• Confounding and data analytics
  • Describe all assumptions and bias that could have influenced the results and its anticipated effect on the direction of the treatment effect estimate.
  • Appropriate covariates selected (i.e., associated with exposure and outcome) and informed via the literature and/or clinical expertise
  • Consider conducting quantitative bias analyses to assess the impact of bias on study results and increase confidence in the results and conclusions.
  • Handle clinically relevant intercurrent events via the estimand framework.

The CADTH Guidance: What’s Missing

The incorporation of RWE studies in decision-making is far from straightforward, however, and critical aspects are missing from the CADTH guidance such as:

• The use of standardized fit-for-purpose quality checklists when multiple RWE sources are considered for a decision problem.
• An understanding of the specific methodological complexities of RWE studies supporting comparative effectiveness claims.
• A closer link to outputs from international pharmacoepidemiology collaborations.

Given the recent use of RWE studies to support technology assessment submissions in other jurisdictions, it will be critical to include guidance for fully capturing critical steps in the generation, synthesis, and critical appraisal of RWE that can establish acceptance thresholds of such evidence and ensure transparency in decision-making related to RWE.

Share Via