Overall survival (OS) is the ultimate endpoint — it is easily measured and of the utmost clinical relevance. However, it also takes the longest time to develop and may not be as sensitive to treatment effect as other endpoints. In many oncology studies, for example, alternative endpoints are used, such as progression-free survival (PFS). The hope is that these alternative endpoints are indeed clinically relevant and allow for quicker results with fewer numbers of events and/or subjects needed than a study powered to detect a difference in OS. However, there has been growing concern over the use of endpoints such as PFS.
As a result, the FDA has pondered guidance on how to prioritize endpoints: whether OS or an alternative endpoint should be primary, or both should be co-primary. The FDA has also pondered whether accelerated approval might be permitted using a quicker-developing alternative while waiting for the OS data to mature and (hopefully) substantiate the efficacious results from the co-primary endpoint.
To address this, the FDA has produced the draft guidance “Approaches to Assessment of Overall Survival in Oncology Clinical Trials.”
Here, we will review this document as it pertains to Data Monitoring Committees (DMCs), and provide our own thoughts based on our 30 years of work and thousands of studies.
FDA draft guidance: “Approaches to Assessment of Overall Survival in Oncology Clinical Trials”
A key section of the FDA guidance related to DMC activity is the following:
This text reinforces what Cytel has long advocated — more interim analyses, both for futility (and possibly harm) and benefit — and reinforces the independence of the DMC and the group directly facilitating the DMC’s work. Approaches to constructing and implementing interim analyses for OS can be flexible — for example, assessing futility at 50% information fraction of OS, and then assessing both futility and benefit at 75% information fraction.
In addition to the section above, there are other references within the draft guidance for how a DMC or other parties would use and interpret OS data. Some particular references are below:
Using and interpreting overall survival data
It is important the DMC understands the operating characteristics of these interim analyses. The analyses might be at a high level in the DMC charter, or more specifically described in the protocol, or study SAP or interim analysis plan (IAP). Translating statistical methodology informally into actual counts is helpful to ensure audiences, including the DMC, understand what the “tipping points” might be. For example, a stopping rule for futility at 50 deaths (with the stopping rule perhaps defined by O’Brien-Fleming or by Bayesian analysis) would helpfully be presented to the DMC that translates to clearly understandable events as a reality check — perhaps the study design and stopping rules might lead to the conclusion that 28 deaths on active vs. 22 deaths on placebo (using reasonable assumptions for timing of events and other aspects of the data) would be the tipping point to trigger futility. This could be informally presented to the DMC at their organizational meeting to ensure the DMC understands the high-level approach and how that might play out in real numbers.
It is also important for the group creating the analyses to communicate to the DMC the uncertainty of early results so that the DMC does not overinterpret. This is especially true in the (theoretic) presence of non-proportional hazards.
Overall survival data: A key safety parameter
OS is a critical variable to help the DMC assess the study even outside of formal interim analyses — both as the DMC interprets safety (more deaths on the active arm) and efficacy (fewer deaths on the active arm). Together, this provides a more complete way to assess risk-benefit in the context of other safety concerns. While some sponsors prefer a “safety only” DMC and are hesitant to provide endpoint data, it is typically well understood that, as a key safety parameter, OS data is
Therefore, we typically present a death table to the DMC, and likely two: one summary of deaths for the safety population and one for the randomized population. This is particularly relevant for open-label studies where there might be an imbalance in subjects not treated and therefore not included in the safety population. Note that, traditionally, not all deaths are included in the adverse event (AE) dataset. In many studies, AEs occurring after treatment has finished and/or due to underlying disease might not be captured in the AE dataset.
Cytel also may tabulate investigator-assessed cause of death. (We do note to DMCs the hazards of interpreting investigator-reported causality though. That is part of the reason OS is appealing, as it is immune to the potential biases of ascertaining relationship.) Two treatment arms might have the same number of deaths, but the interpretation of the DMC could be different if there is an excess of deaths due to disease progression on one arm, relative to an excess of deaths due to serious adverse events (SAEs)/toxicity on another arm. It is traditional to also show timing in a table (perhaps deaths <30 days after last dose vs. deaths ≥ 30 days after last dose). The DMC may drill down into the fatal SAEs/toxicity to look for patterns and perhaps make recommendations to mitigate that risk. We have seen examples of excess fatal interstitial lung disease (ILD), fatal COVID-19, and fatal infections on active arms compared to control arms. DMCs in these situations have thought hard about mitigation strategies, as they would for any concerning imbalance in a major safety domain.
A Kaplan-Meier figure of time to death (possibly without any inferential statistics) is also typically presented to the DMC — typically based on the randomized population. It can be important for the DMC to understand the timing of the deaths to help answer questions such as whether most deaths occur early in the study or if there is a differential pattern over time (i.e., crossing curves implying non-proportional hazards) between the treatment arms. The definitions used for programming the Kaplan-Meier figure might not match the exact definitions at end of study if used in a non-inferential way — the censoring rules might be flexible based on the data available (e.g., the choice of whether to censor subjects still alive and in study follow-up at their last known contact, or whether to censor at the data cut-off date). In some situations, especially where OS is the primary endpoint, the sponsor might be hesitant to even show a non-inferential Kaplan-Meier figure of time to death to the DMC outside of formal analyses (if those exist). Nonetheless, the DMC could argue these are needed and, at minimum, are available upon request by the group facilitating the DMC’s discussion.
For formal interim analyses of OS that show inferential statistics (e.g., p-values, hazard ratio, confidence intervals), there would be appropriate effort in advance to ensure that the precise censoring algorithm and inferential statistics specified in in the study SAP are put in place for the DMC review.
Oversight on study integrity and interpretability
The DMC has oversight on study integrity and interpretability in conjunction with the sponsor, but it is important to have the independent thoughts of the DMC. In this domain, the DMC may well voice concern if an excessive rate of subjects is being lost to long-term survival surveillance. That would be particularly important if an imbalance in arms is developing for subjects lost to follow-up for long-term survival (as could happen in open-label studies). The DMC might not explicitly state that observation externally, but an imbalanced rate of lost to follow-up would be particularly concerning to the interpretation of OS at the end of the study.
And the DMC may voice concern if OS is the primary endpoint or it is critical to have a certain number of deaths for secondary analysis, but the overall rate of deaths is appreciably lower than expected, which seems likely to extend the duration of the study by years – bringing into question whether the study will be relevant and funded for the additional years needed. The DMC may request current projections (with computed confidence interval) for when in calendar time the expected number of deaths will occur. This can be complicated if the minimum number of deaths is required in a subgroup, and particularly so if the minimum numbers of deaths is in a blinded subgroup (a blinded biomarker population, or in a subgroup of treatments for studies with more than two treatment groups).
The DMC should understand that OS data early in a study is statistically unreliable. And there might even be some expectation for excess deaths early on (if late benefit is expected, in the presence of early toxicity). There might even be the expectation of no OS benefit — there might still be overall value to the new treatment even if there is no difference in OS — if the treatment is less toxic or more easily administered or cheaper.
Increasing the confidence of subgroup analysis results
The DMC may rightfully be concerned upon seeing an excess of deaths on the active arm. There are options for the DMC, as for any potential safety concern. One of the first steps traditionally undertaken is to investigate which subgroup of patients is most at risk for the excess death — but with the full understanding of the hazards of subgroup analysis. It is easy for an unwary reviewer to overinterpret the results of subgroup analyses. The DMC should consider factors that increase the confidence of subgroup results when looking for consistency of signal or trying to identify a subgroup that could specifically have some risk mitigation plan implemented:
- Biologic rationale
- Larger sample size
- Consistent findings in other trials
- Included as a stratification factor
Ad hoc interim analyses for OS
If OS is the primary endpoint, and no statistical futility in place but the DMC sees worrisome or neutral trends in OS, the DMC might consider ad hoc analysis of OS for futility. This is usually in the context of excess Grade 3+ AEs or SAEs as well as this OS result. The DMC might see if a lower limit of a confidence interval of hazard ratio (suitably adjusted for the interim nature by using information fraction and perhaps O’Brien-Fleming boundary) excludes 0.9 or lower. Or perhaps the DMC will request the supporting CRO to compute conditional power. We have seen DMC recommend termination or major change based on the totality of the data — largely influenced by neutral or negative OS results. Obviously, it is preferred to simply have pre-planned looks for OS futility.
The DMC might also see amazing benefit of OS, but without any formal interim analysis planned to assess benefit. The DMC could decide that the scientific question has been answered and that it is unethical to continue and therefore alert the sponsor. We have seen this occur and after intense focused discussion between the DMC and sponsor and regulatory agencies the decision was made to move forward towards regulatory approval, which accelerated the regulatory process by years. Clearly, it is preferred to simply have pre-planned looks for OS benefit.
A recent situation involved a relatively high toxicity observed, but the intervention was expected to provide a long-term OS benefit. The DMC decided to recommend halting enrollment in the population with the lowest baseline disease severity. These subjects were expected to have the same rate of toxicity as all groups, but with minimal expected absolute OS benefit.
Difficult recommendations
DMCs must make difficult recommendations as PFS and OS data emerge, if PFS is a primary or co-primary endpoint. The DMC should ask for and be provided with (perhaps non-inferential) summaries of both PFS and OS. DMC recommendations are challenging if results are discrepant. Most common would be an interim analysis of PFS for benefit that crosses a boundary for benefit, but OS results are extremely immature or neutral or perhaps even in the negative direction. The DMC hopefully has the flexibility to explain the context of the situation to a senior liaison at the sponsor for discussion if the study should continue and obtain additional valuable OS data to help answer questions about OS more precisely. This is particularly problematic if there is a discrepancy between investigator-provided PFS and blinded independent review committee (BICR) PFS, or a delay or unavailability of BICR data to the DMC.
Controversies exist when the co-primary endpoint achieves full statistical information before OS does and deciding what the DMC’s role is at the point, and after that point in time. In some situations, the DMC’s obligations for oversight of the study conclude once PFS has completed. The sponsor (perhaps the full study team, or a subset of the study team, or a separate team) becomes unblinded to final PFS (and likely interim OS), and oversight for safety of ongoing patients during continued OS surveillance is undertaken by the sponsor. The assumption is that unblinding and knowledge of PFS results will not bias the ongoing OS collection. However, if results of PFS are impressive and that knowledge becomes public, that might impact behavior that affects future collection of OS — especially in an open-label study.
Therefore, many DMCs have argued that the DMC should still be involved in oversight of OS even after final analysis of PFS — perhaps in conjunction with making sure that the final PFS results are handled very securely and led by DMC and not sponsor personnel. The value of this approach is that the continued OS is not impacted by patients, sites, or general sponsor team knowledge of the PFS results or interim OS results. The DMC can take the lead on communicating final PFS results and perhaps interim OS results to a small group within the sponsor or perhaps (with agreement from all parties) directly to regulatory agencies.
Final takeaways
We believe, and the FDA agrees, that DMC access to overall survival results can be critical to the DMC’s remit. Hopefully, DMC members will insist on receiving the outputs that meet their needs, which could include either non-inferential or inferential analyses of overall survival; CROs will be able to create the outputs needed in an accurate and timely way; and sponsors will trust DMCs to act responsibly with the outputs provided to them.
Interested in learning more?
Download our white paper, “DMCs for Oncology Studies”:
