ITCs are essential for comparing two or more interventions when head-to-head randomized controlled trials (RCTs) are unavailable. In cases where patient-level data are available for at least one study, population-adjusted methods, such as matching-adjusted indirect comparison (MAIC) or simulated treatment comparison (STC), can be used to adjust for differences in treatment effect modifiers (TEMs) and prognostic variables (PVs) across study populations. Typically, relevant TEMs and PVs are identified through literature reviews, statistical approaches, and expert opinion.

To ensure the accurate identification and clinical relevance of TEMs and PVs, sponsors often consult with key opinion leaders (KOLs), improving the reliability of the ITC results. However, despite the critical and nuanced insight they can provide, the role of KOLs remains unstructured in formal guidance.

Here, we discuss the need for guidelines that outline when and how to integrate KOL input for better-informed healthcare decision-making.

The uncharted role of KOLs in ITCs

Health technology assessment (HTA), specifically in the context of ITCs, is an area driven largely by quantitative methods. Yet, the qualitative nuance provided by clinical experts, or KOLs, is indispensable. KOLs offer insights into TEMs and PVs that might otherwise elude purely data-driven models.

Current guidance documents

Limited structured guidance from HTA bodies and professional societies leads to inconsistent KOL engagement. Current guidance documents for conducting ITCs (e.g., the NICE methods manual1 and DSU TSD 182) emphasize identifying potential TEMs via expert discussion but leave practical KOL engagement strategies underdefined.

Guidance from non-payer organizations

The lack of guidance from non-payer organizations is also evident. The Cochrane Handbook3 discusses both methodological strengths and pitfalls of potential bias when treatments or populations vary in subtle but clinically important ways. However, the guidance is primarily focused on quantitative data synthesis rather than the integration of qualitative insights.

The PRISMA guidelines4 stress the need for transparent reporting when combining evidence across comparisons, but no tools or frameworks are proposed for capturing qualitative contributions.

A 2023 review5 of methodological approaches for identifying TEMs in ITCs highlighted that available guidance largely focused on statistical methods for adjusting TEMs rather than systematic and comprehensive processes for identifying TEMs. In addition, only 17 of 511 (3.3%) ITCs included in the review presented a description of the selection process for TEMs.

Bridging the gap: Methodological and procedural challenges

The current landscape is marked by a reliance on ad hoc approaches. KOL input on ITCs is often collected on an “as-needed” basis — sometimes late in the process — which may result in missed opportunities to refine study protocols early on. Without structured guidance from HTA bodies and professional societies, engagement with KOLs remains inconsistent. This gap underscores a broader tension: the need to honor the nuance of clinical insights while adhering to statistical rigor. It has been suggested that solicitation of KOL input should occur during the early, formative phases of the research process within a pre-specified framework.6 Late-stage involvement and integrating KOL input post-hoc  — after the core design and analysis decisions have been made — can introduce bias and risks the integrity of the ITC.

Several methods for structured expert input, such as the Delphi technique or nominal group processes, have been proposed in adjacent fields, but these are rarely applied in the context of ITCs.7 The consequence is a reliance on unstructured interviews or informal consensus-building, which can introduce subjectivity and reduce reproducibility.

Looking forward: Harmonizing expert input with methodological standards

The way forward lies in bridging the divide between clinical intuition and methodological precision. The development of clear guidelines that outline when and how to integrate KOL input would be a significant step toward enhancing the reliability of ITCs. One promising approach is the early engagement of KOLs in, for example, structured Delphi panels or advisory boards during the protocol development stage. Codifying this process would ensure that expert insights inform the research from the outset, rather than serving as an afterthought.

HTA bodies, regulatory agencies, and academic methodologists should prioritize the collaborative creation of comprehensive guidelines to address the following key aspects of KOL consultation for ITCs:

• Selecting KOLs: Defining objective, transparent eligibility criteria to ensure that only the most appropriate clinical experts contribute their insights.
• Timing: Detailing when during the research process expert opinion should be solicited — ideally early on to influence study design and TEM identification.
• Question types: Guiding the formulation of questions that address specific knowledge gaps related to TEMs and PVs in the ITC.
• Integration protocols: Outlining systematic methods for incorporating and reporting KOL insights, ensuring this information is subject to the same transparency standards as quantitative data.

The accurate identification of TEMs can significantly influence the reliability of ITC outcomes. Without consistent frameworks, there’s a risk that KOL input is either underutilized or inconsistently applied, affecting both the credibility and applicability of findings. The lack of guidance highlights the need to explore effective approaches in the absence of standardized methodologies for KOL engagement. Addressing this gap is essential for improving the reliability of ITC results and supporting informed healthcare decision-making.

Upcoming webinar

Our Evidence, Value, and Access team will be hosting the upcoming webinar, “Navigating the First Year of EU JCA Implementation: Updates, Methodological Insights, and Bridging Local HTA Realities,” on July 10 at 10 am ET. Register today to reserve your spot!

Share Via