Randomized controlled trials (RCTs) remain the gold standard for the evaluation of the safety and effectiveness of a new treatment. However, in a number of cases alternative approaches leveraging external data (i.e., data from outside of a clinical trial) — ranging from single arm trials to augmented RCTs — can be appropriate. Here, we discuss how to leverage and incorporate external data in drug development, focusing on the use of external control arms and Bayesian borrowing.
Behind the Oncology Research: An Interview Between Robert Szulkin and Jana de Boniface
Welcome to our interview with Dr. Jana de Boniface, a renowned surgeon and researcher specializing in breast cancer. In this conversation, we delve into her inspiring journey, groundbreaking research, and collaborative efforts that have led to significant advancements in breast cancer treatment.
Robert Szulkin (RS): Jana, you’ve just published a paper on “Omitting Axillary Dissection in Breast Cancer with Sentinel-Node Metastases” in the New England Journal of Medicine (NEJM) [1]. Congratulations! This project was completed in cooperation with Cytel’s Real-World Evidence team, and we’d love to discuss the process that made this project a reality. But first, a brief introduction. You are a surgeon specializing in oncoplastic breast surgery as well as a Professor in the Department of Medical Epidemiology and Biostatistics at the Karolinska Institute in Sweden. I’d like to ask a bit about your background.
What inspired you to become a scientist and a surgeon?
Jana de Boniface (JdB): I began my journey as a scientist during my university years in Berlin, and that was just a completely different topic, it was psycho-oncology. When I moved to Sweden to train as a surgeon, I didn’t have a specific moment of inspiration. However, a fantastic senior colleague, Leif, approached me early in my training and encouraged me to pursue research. He must have seen something in me—perhaps my curiosity and perseverance. Since then, what drives me to continue with research is the patients and seeing them in situations where I cannot say why I am doing what I am doing. I need to have evidence for the treatments I recommend. For instance, if I tell a patient they need a mastectomy or need lymph nodes removed and I know it’s going to hurt the patient in the long term, I want to be certain that these actions are necessary. It’s basically this unhappy feeling in the clinic when I just don’t know if I’m doing the right thing.
About how I became a surgeon, I was going to become a psychosomatic doctor, which is a combination of internal medicine, psychology, and psychotherapy in Berlin. Everything was planned: exams, research, post-doc work. Then, during my final obligatory year as a student, I decided to go to Sweden for my mandatory surgical training, as my best friend Michaela had moved there. During these three or four months at the clinic, I just felt like I was home. The hands-on nature of surgery, the immediate impact of the work, and the ability to cure people felt incredibly appealing. My mother, a psychologist, once told me, ‘ You can’t cure people with a knife’ but I believe you can!
RS: That’s interesting. I thought you always wanted to be a surgeon. I’ve seen you perform surgery, and it was incredibly inspiring. What was the first research question you began to investigate in your career, and how has this changed over time?
JdB: My initial research in Berlin was completely different—it was a study on the treatment of hepatitis C. However, I set that aside. My first projects involved sentinel lymph node biopsy in breast cancer, around the early 2000s. At that time, we still performed many axillary clearances, so we started implementing sentinel lymph node biopsies while still performing clearances to ensure the sentinel nodes weren’t giving false-negative results. That’s how I started. We were still discussing the performance and accuracy of sentinel lymph node biopsy in large tumors as a different project. I also delved into tumor immunology, studying the immune response in lymph nodes, which was quite exciting.
RS: So, you’ve been doing axillary surgery research for almost 25 years now. That’s quite impressive. I guess you know the subject very well after all this time.
JdB: Yes, I’ve been deeply involved in it for many years.
RS: Let’s talk about our recent publication, which is a very hot topic right now. The results have been presented worldwide. The research conducted aims to reduce the burden of surgery in breast cancer patients, such as the invasiveness of surgery and potential complications. And now this research has resulted in the first surgical paper to have been published in NEJM in the last 17 years. The SENOMAC trial started back in 2015; how did you come up with the idea to start this study?
JdB: It was of course not just my idea, but we were a team from the beginning. When we started the study, we thought it would just confirm the results of two previous studies conducted in the US and the Netherlands, which had shown similar outcomes. But, those studies faced a lot of criticism, especially the US one, due to their small sample sizes and lack of statistical power. They weren’t convincing enough. Back then, I argued against continuing axillary dissections but the Swedish National Guidelines committee deemed the data too weak to support stopping them. So, we kept performing axillary dissections, which felt frustrating. We were then a small group of people who decided we’d try to start a trial adding more data to the area and broadening the population characteristics to also embrace new questions. For me, this research was a tool to reduce the number of dissections because if guidelines don’t allow you to do less, you need to research to prove the point, and at the same time, research allows patients to get access to a more modern approach.
Before our study was complete, the Swedish guidelines changed, allowing the omission of axillary dissections for certain patients, and replacing it with radiotherapy. So, I didn’t expect our study to have such a significant impact. While it doesn’t change the Swedish guidelines, it significantly influenced guidelines in other countries. I’ve received many inquiries about the inclusion criteria, such as whether we included patients with extra-nodal extension and if our findings apply to various patient types. And then I noticed that people have probably not just used these older trial data in their full scale, but they’ve used it for some subgroups. They still hesitate to omit axillary dissection in certain types of patients. And I think our study, because it’s so large and has full statistical power, is the first to provide definitive answers to these questions. That’s fantastic and it’s gratifying to see its impact.
RS: Where is this already implemented? What kind of impact will this research have?
JdB: Before the study, I believed that many countries already omitted axillary dissection. The big question was regarding mastectomy patients, those undergoing removal of the entire breast, because we lacked data on that group. The impact of our study is significant here.
Some large countries had already implemented omitting axillary clearance, but they didn’t apply it to patients where metastasis had grown outside the lymph node. After our study, they began to include these patients as well, seeing similar outcomes. Many guidelines adapted these older trials with specific limitations, and I think our study helped to remove those limitations. While I don’t know every single country’s guidelines, I believe many were already moving towards reducing axillary dissection for more subgroups.
In the US, the first trials on omitting axillary clearance in breast-conserving surgery were conducted around 2011-2012. They allowed the omission of axillary clearance in these cases but not in mastectomy patients. It appears their guidelines suggest considering it, but it wasn’t standard practice. Our study is now providing the necessary evidence to support broader implementation, including for mastectomy patients.
RS: How would you describe the results of your research to a non-clinician, such as a patient? What kind of impact could this research have on them?
JdB: For a patient, I would explain it like this: Normally, if we don’t see any signs that cancer has spread to the armpit, we remove something called a sentinel lymph node. This is the first lymph node in the armpit which receives lymphatic fluid from the breast, and sometimes, cancer cells if the tumor spreads. We typically remove one, two, or three of these nodes. And then, if we find signs of metastasis in these nodes, previously, we used to perform an additional operation to remove more lymph nodes from the armpit—usually more than 10. This often led to problems with the arm, such as swelling, pain, and restricted movement in the arm, which could last a long time.
Now, our research shows that removing more lymph nodes does not improve survival rates or reduce the risk of cancer recurrence. So, we can safely leave the remaining lymph nodes in place even if the disease may be present in these nodes. This makes a big difference in the patient’s recovery and quality of life.
RS: You participate actively in real-world evidence studies, observational studies, and clinical trials. Why do you think that is important? Can they complement each other, and what’s your view on that?
JdB: In my field, many questions will never be addressed in randomized trials. For example, deciding whether to perform a mastectomy or a lumpectomy, or whether or not to have an immediate breast reconstruction, can’t be left to a coin toss today. Patients need to be involved in the decision-making process, weighing the pros and cons to make informed choices.
For issues like breast conservation versus mastectomy, new randomized trials beyond those performed in the 70s and 80s are unlikely. Instead, we rely on high-quality, population-based databases with comprehensive and reliable data. These databases allow us to adjust for all these confounders that we know we have and provide insights that randomized trials cannot. There are many similar questions where patient choice is paramount, such as whether to undergo breast reconstruction.
Prospective cohort studies also play a crucial role. If there’s evidence suggesting a new method might be better, but the existing trials are not conclusive enough, we can implement this method in a prospective cohort study. This allows us to monitor patients closely and ensure they receive modern treatment while still being able to assess the method’s effectiveness. If the method turns out to be suboptimal, we can identify this and adjust accordingly.
RS: As I mentioned earlier, this project was completed in cooperation with Cytel’s Real-World Evidence team. Why did you choose to partner with us on this project?
JdB: We specifically chose to work with you because our colleague, Anna Johansson, and I discussed the idea of breast conservation versus mastectomy. When we needed someone for the hands-on statistical analysis, Anna recommended you because of the previous positive experience and partnership we have had.
Before we started our collaboration, I mostly did statistics myself. However, with increasing study sizes and the need for randomized trials, I felt the quality had to be completely watertight, and you provided that. I also quite like our collaborative discussions and your proactive ideas. You’re not just executing tasks; we have a dialogue and we brainstorm our options and come up with solutions to the problems that arise throughout the process. Our team, including Anna, shares a commitment to timelines and schedules, making us very effective. When we wrote the articles, we successfully streamlined the process and ensured everyone contributed and helped.
RB: I agree. Our close collaboration and communication work very well. You explain the clinical aspects clearly, and I can, most of the time, explain the statistical context of everything. Even tasks like data cleaning and checks, which could be tedious, were smooth due to our excellent communication. It’s wonderful to have a good working relationship with sponsors, and it’s been a great experience working with you and Anna.
Interested in learning more about Cytel’s Real-World Evidence solutions? Read more.
Note:
Planning Strategies for Externally Controlled Trials: Insights from ISPOR US 2024
External Control Arms (ECAs) provide comparative evidence when recruiting patients is difficult or unethical in randomized controlled trials. ECAs have significant potential to save resources and accelerate access to innovative treatment. In a previous blog, our experts took a deep dive into the concept of ECAs, their acceptable use cases, and the current regulatory guidance.
Existing guidance on the design and conduct of externally controlled trials emphasizes the importance of early engagement with regulatory and HTA bodies to justify using an ECA and discuss the preliminary study design and statistical analyses. With the increasing use of ECAs in regulatory and HTA submissions, the acceptable use cases for ECAs and important design and analytical considerations are becoming clearer. However, sponsors still face key questions about the optimal timing to plan for an ECA and how to prepare for early interactions to address differing regulatory and HTA perspectives.
In the ISPOR US 2024 HEOR Theatre session, Jason Simeone, Evie Merinopoulou, and Grace Hsu delved into these questions, discussing how regulatory and HTA stakeholders appraise ECAs, common issues from both perspectives and proposed practical solutions. In this blog, we ask Evie follow-up questions, highlighting insights from their ISPOR HEOR Theater session.
Your ISPOR US 2024 presentation was about early planning strategies for ECA. So, what is the optimal timing to start planning for an ECA?
Ideally, sponsors that need to perform an ECA to support their development program should start planning for the ECA alongside the clinical trial design. This allows them to gain experience with current real-world data (RWD) and make any necessary investment decisions for data improvements, such as additional data collection or infrastructure upgrades. Further, considering an ECA during the trial design provides the opportunity to incorporate real-world endpoints into the clinical trial. This is particularly valuable because defining clinical endpoints in real-world databases can often be challenging, especially when they are not measured consistently between routine practice and clinical trials.
Further, we showed in our presentation how although formal guidance from regulatory and HTA bodies on ECAs is consistent, final decisions when appraising ECAs may differ. This divergence in regulatory vs HTA acceptance reflects differing requirements for ECAs. When planning ECAs, both perspectives and requirements should be considered. Therefore, within sponsor organizations, early planning is key for cross-functional alignment (between HEOR/Market Access and Medical Affairs teams) on ECA study objectives and design, leading to more efficient evidence planning. With regards to external engagements with regulators and payers, the optimal timing is very contextual, but generally, sponsors should engage with decision makers via available routes like early advice programs, early enough to have the time to incorporate feedback and adjust their RWD strategy and study design—before protocol and SAP finalization.
Is early planning necessary for all cases? For instance, if a product is being developed for an indication with a rapidly changing treatment landscape and the appropriate comparators may not yet be known, would these early planning activities still be useful?
Yes, absolutely. During the early feasibility assessments that we discussed in our ISPOR presentation, we should evaluate a range of elements to determine the feasibility of an ECA—ranging from the identification of target populations to the reliable capture of confounders and study endpoints, among other factors. Identifying relevant comparators is only one element of those assessments. Even if comparators change over time, becoming familiar with RWD and current gaps helps inform discussions about the appropriate data strategy and design, which should be flexible enough to reflect some of the changes in the treatment landscape. Perhaps now, we would want to know if treatments are well captured and elements like patient count on a relevant comparator will need to be refreshed. It is important to ask questions during the early planning stages that are specific yet broad enough to inform ECA feasibility, even if the research question evolves, particularly concerning the RWD strategy.
You’ve recommended that study sponsors should be prepared to discuss certain topics during early engagement meetings, such as the ECA rationale, data source, early design considerations, and feasibility assessment. In a resource-constrained environment, sponsors may not want to invest so much money in these activities before the very first meeting, only to receive a negative response. What topics should be prioritized for that first engagement with an HTA or regulatory agency vs. subsequent meetings?
This is an important point. Ultimately, the most crucial aspect is to clarify the justification for an ECA and assess whether the agencies are open to considering evidence from an ECA. Working with the right experts who understand agency requirements from prior experience is important. Beyond the justification for an ECA being clear, we see that most critiques of ECAs stem from data issues. So, in my opinion, presenting external data source options and discussing anticipated challenges can facilitate a more productive discussion in those early engagements. If resources are constrained, a more targeted review is sufficient rather than a full-blown data landscaping exercise.
During your presentation, you emphasized the importance of identifying fit-for-purpose data. However, in some cases, a sponsor may have to submit to an HTA body in a region where such data is not readily available. For instance, if a detailed data landscaping assessment reveals that most fit-for-purpose data is in the US, but the submission is for a European HTA agency, how can sponsors address this challenge in their submission?
First and foremost, sponsors need to present to the local agency that they have thoroughly attempted to identify a data source accurately representing the local (target) population of interest. Local agencies are usually quite understanding if sponsors can demonstrate that they made the necessary effort and did not cherry-pick data sources, but instead selected a source with the highest quality data available for the research question, in a transparent and systematic manner. However, this means that there might be some potential external validity bias that could concern a local decision-maker. For example, a UK or German payer might be concerned that evidence submitted from a US data-derived ECA may not be generalizable to the target population of the decision problem.
At Cytel, we have been engaged in some very interesting work to understand how we could adjust for this potential external validity bias using transportability methods. These are quantitative methods, similar to those adjusting for confounding, and can be reliably used to extend conclusions from one study population to an external target population. Essentially, if core evidence comes from a US-data derived ECA, transportability methods can be applied to adjust the study findings to measurable patient characteristics in the target population of interest, accounting for prognostic factors or effect modifiers. We recently published a demonstration project on this topic [1]. Additionally, NICE recently updated its RWE framework [2] to include transportability analysis methods.
Alternatively, sponsors could consider designing a prospective study, though this approach requires much higher costs and extensive timelines. If you’re taking this route, you should design data collection with the ECA in mind, aligning patient selection criteria, endpoint definitions, etc., which is why planning early is important.
Overall, at Cytel we encourage sponsors to approach data selection in a transparent and systematic way, as recommended across all existing ECA formal guidance documents, and leverage available analytical approaches to address potential external validity concerns when using non-local data if additional data collection is not feasible.
Which internal stakeholders should be involved in this process of early planning for ECAs, and what should sponsors consider when partnering externally?
Typically, in sponsor organizations, there are clinical development and medical affairs teams that understand regulatory requirements and processes very well. In addition, there are Market Access and HEOR/RWE teams that know RWD and real-world evidence methods very well. These teams may not always work closely together, but in our presentation, we talked about the importance of bringing these two teams together early on in planning for ECAs to align differing regulatory vs payer requirements. When selecting external partners, it’s important to work with organizations that have important methodological and technical expertise. They should also have a thorough understanding of the evolving guidance and acceptance criteria of decision-making agencies and be able to provide strategic guidance on important study design decisions and early stakeholder engagements.
Interested in exploring further? Download the slides from the ISPOR HEOR Theatre Session presented by Cytel here.
Notes
[1] Ramagopalan SV, Popat S, Gupta A, et al. Transportability of Overall Survival Estimates From US to Canadian Patients With Advanced Non–Small Cell Lung Cancer With Implications for Regulatory and Health Technology Assessment. JAMA Netw Open. 2022;5(11):e2239874. doi:10.1001/jamanetworkopen.2022.39874
[2] https://www.nice.org.uk/corporate/ecd9/resources/nice-realworld-evidence-framework-pdf-1124020816837
Unravelling PICO: The Pillars of the European Joint Clinical Assessment
The European Union (EU) health technology assessment (HTA) regulation aims to improve the availability of innovative technologies for patients across the European Union (EU).1 It is also claimed to offer efficiency gains for manufacturers, due to one single EU-level submission vs. multiple parallel submissions to different HTA bodies.2 In this blog, we will first introduce the Joint Clinical Assessment (JCA), a legal requirement of the EU HTA regulation; the pillars that hold the JCA together, the PICO framework; and the consequential impact to manufacturers on reporting requirements based on multiple PICO.
Using Quantitative Bias Analysis in Real World Data Strategy
The gold standard for assessing the efficacy for a medicine continues to be RCTs, however, for many reasons (disease rarity and/or ethical concerns), two-arm trials with adequate power may be infeasible. In such cases, single-arm trials or a purely observational study are conducted. To evaluate comparative treatment effects using data from single-arm trials, non-randomized studies, or indirect treatment comparisons (ITCs), are designed to incorporate external data sources. These include datasets from other historical trials or observational, real-world data (RWD).
What it means to be a lead analyst on a Global COVID-19 Trial
The TOGETHER Trial for COVID-19 therapies, designed by clinical trial specialists at Cytel won the Society for Clinical Trials David Sackett Trial of the Year Award for 2021. I interviewed Hinda Ruton, Research Associate at Cytel, who made significant contributions as Lead Analyst to several studies in TOGETHER Trial.
Hinda has been working as a lead statistician and statistical programmer for clinical trials, and statistician for real world data (RWD) analysis. Prior to joining Cytel, he was the program coordinator of the Rwanda Human Resources for Health program in the Ministry of Health. He coordinated a program of $150 million USD, aiming to build the capacity of health professionals. This was done in collaboration with 25 of the best teaching institutions in the US. Hinda has an academic appointment at the University of Rwanda where he teaches biostatistics and information management systems.
Writing a Successful Study Protocol for Real-World Evidence Studies
Real-world evidence studies are becoming increasingly popular in pharmaceutical development. But to ensure such studies are feasible and of high scientific quality, a well-written study protocol is essential. Let’s take a closer look at how to write a successful study protocol for real-world evidence studies: Read more »
The New EU HTA Landscape: Insights on Indirect Evidence
How should health technology developers prepare for future market access activities in Europe?
Numerous discussions have already taken place in various forms and on various platforms around the upcoming implementation of the EU Joint Clinical Assessments (JCA); it’s a hot topic and keeps many of us in our industry occupied. Despite the European Commission’s active efforts in developing draft regulation and related materials to support the transition to JCA, to date, some questions remain unanswered. As the EU JCA aims to harmonize and accelerate evaluation processes in Europe, all stakeholders, including health technology developers, national health technology assessment (HTA) authorities, and EU JCA assessors and patients, are facing substantial changes with this new process on European-level HTA.
U.S. Drug Pricing Reform: Potential Impact on Pharma HEOR Evidence Generation
On August 16, 2022, President Biden signed into law the Inflation Reduction Act of 2022, which includes U.S. drug pricing reform that, among other things, requires the federal government to negotiate prices for some high-cost drugs covered under Medicare. In our recent webinar, Vice President of Value and Access Anna Forsythe discusses the effect that a strong HEOR strategy can have to help sponsors navigate this space for the benefit of all parties.