Annals of Oncology Advance Access originally published online on February 27, 2008
Annals of Oncology 2008 19(7):1347-1353; doi:10.1093/annonc/mdn042
clinical trials |
Stopping a trial early in oncology: for patients or for industry?
1 Italian Medicines Agency (AIFA), Rome
2 Mario Negri Institute for Pharmacological Research, Milan, Italy
3 Utrecht University, Utrecht Institute for Pharmaceutical Sciences, Utrecht, The Netherlands
* Correspondence to: Dr G. Tafuri, Italian Medicines Agency (AIFA), via della Sierra Nevada 60, 00144 Rome, Italy. Tel: +39 6 59784713, Fax: +39 6 59784717; E-mail: g.tafuri{at}aifa.gov.it
 |
Abstract |
|---|
 Top Abstract backgroundobjectivematerials and methodsresultsdiscussion and conclusionsAcknowledgementsReferences |
|---|
Background: The aim of this study is to assess the use of interim analyses in randomised controlled trials (RCTs) testing new anticancer drugs, focussing on oncological clinical trials stopped early for benefit.
Materials and methods: All published clinical trials stopped early for benefit and published in the last 11 years, regarding anticancer drugs and containing an interim analysis, were assessed.
Results: Twenty-five RCTs were analysed. The evaluation of efficacy was protocol planned through time-related primary end points, >40% of them overall survival. In 95% of studies, at the interim analysis, efficacy was evaluated using the same end point as planned for the final analysis. As a consequence of early stopping after the interim analysis, 
3300 patients/events across all studies were spared. More than 78% of the RCTs published in the last 3 years were used for registration purposes.
Conclusion: Though criticism of the poor quality of oncological trials seems out of place, unfortunately early termination raises new concerns. The relation between sparing patients and saving time and trial costs indicates that there is a market-driven intent. We believe that only untruncated trials can provide a full level of evidence which can be translated into clinical practice without further confirmative trials.
Key words: anticancer drugs, EMEA, FDA, end point, interim analysis, RCT
|
background |
|---|
|
TopAbstractbackgroundobjectivematerials and methodsresultsdiscussion and conclusionsAcknowledgementsReferences |
|---|
European legislation in pharmaceuticals has been recently revised. In line with the path of the United States Food and Drug Administration (FDA), new procedures for granting marketing authorisation now include accelerated and conditional approvals, leading to quicker access of new drugs to patients. In this evolving scenario, guidelines on the evaluation of medicinal products are subject to continuous revision. This is especially the case for anticancer drugs, for which important changes have been made in terms of trial design and conduct [1]. These factors have simplified and shortened the process of development of a new drug, particularly in oncology.
Previous analyses of new anticancer compounds, approved by both the European Medicines Agency (EMEA) and the FDA, highlighted methodological concerns in terms of lack of comparative trials, use of surrogate end points, lack of evidence for establishing the added value, and lack of blinding/masking [2–5]. In addition, a recent systematic review covering different therapeutic areas found that the number of randomised trials stopped early for benefit had more than doubled since 1990 [6].
Interim analyses pose the ethical dilemma of safeguarding the interests of patients enrolled in clinical trials while also protecting society from overzealous premature claims of treatment benefit. Trials stopped early because of harm (toxicity) or futility tend to result in prompt discontinuation of useless or potentially harmful interventions. In contrast, trials stopped early for benefit may result in the quick identification, approval, and dissemination of promising new treatments.
Given the serious and life-threatening nature of cancer and patients’ expectations, quicker clinical drug development is required by both patients and clinicians, but this may lead to an unclear and poorly defined benefit/risk balance of new drugs.
|
objective |
|---|
|
TopAbstractbackgroundobjectivematerials and methodsresultsdiscussion and conclusionsAcknowledgementsReferences |
|---|
The aim of this study is to assess the use of interim analyses in randomised controlled trials (RCTs) testing new anticancer drugs, focussing on oncological clinical trials stopped early for benefit. A second aim is to estimate how often trials prematurely stopped as a result of an interim analysis are used for registration purposes. Our study presents an updated overview of this growing phenomenon in the specific field of oncology, which is subject to continuous change. The analysis focussed on trials that were halted after an interim analysis found the treatment carried out better than the control arm.
|
materials and methods |
|---|
|
TopAbstractbackgroundobjectivematerials and methodsresultsdiscussion and conclusionsAcknowledgementsReferences |
|---|
All clinical trials published from January 1997 to October 2007, regarding anticancer drugs and containing an interim analysis, were retrieved through Medline. The following strategy was adopted: publications containing the words ‘interim’ and ‘analys*’, and limited to humans, clinical trials, cancer, and English language, were searched. A total of 231 reports were found.
In order to test the sensitivity of the research methods, we did an extra search for articles, that might have been missed in the first search, published in the three main peer-reviewed journals (The Lancet, The New England Journal of Medicine, and The Journal of Clinical Oncology) from October 2006 to October 2007; this produced two more reports. These three journals were chosen on the basis of how frequently they had reported the articles retrieved through the Medline search.
To increase the specificity, articles were initially screened on the basis of the abstract. Of the 233, 140 reports were excluded as not relevant according to predefined inclusion and exclusion criteria (see Figure 1). In particular, phase I trials, trials testing growth factors, and those based solely on surgery or radiotherapy were considered not pertinent, so were excluded. Furthermore, studies comparing different dose regimens and schedules of the same drugs and studies on the basis of palliative/supportive therapies (e.g. antiemetics) were excluded. Study protocols were also excluded.
|
Only papers on the basis of trials of anticancer medicinal products and containing an interim analysis were initially considered eligible for analysis (93 papers describing 93 trials). Out of these 93 papers, 65 were subsequently excluded for the following reasons: 4 trials were stopped after an interim analysis because of harm (toxicity) and 28 because of futility (lack of efficacy). Another 33 papers were excluded because the trials were not actually stopped after the interim analysis and were thus considered ongoing.
Twenty-eight papers met the inclusion criteria, i.e. clinical trials testing anticancer medicinal products truncated for benefit after an interim analysis. However, one was unretrievable, and in two separate cases two papers reported the same study in two different journals. In both cases, only the paper published earlier was included. Following these corrections, a final sample of 25 papers describing 25 trials was obtained.
For the purpose of this analysis, the following parameters were assessed: disease, study duration, date of publication, presence of a ‘Data and Safety Monitoring Committee’ (DSMC), type of end point(s), sample size, rationale for interim analysis and type of analysis carried out, consequences of the interim analysis on the RCT and on the patients, and characteristics of the control group. We reported both the primary end point planned for final efficacy analysis and the end point used for the interim analysis. The same was done for sample size. In a few cases, certain study characteristics were not reported in the data acquisition form because they were not specified in the analysed articles. The investigators defined a priori a common data acquisition form to be completed. FT and GT independently evaluated all the selected papers and filled the respective forms. The results were then cross-checked, leading to a joint document. In the case of disagreement, the final decision was taken through a consensus process reached following a discussion.
|
results |
|---|
|
TopAbstractbackgroundobjectivematerials and methodsresultsdiscussion and conclusionsAcknowledgementsReferences |
|---|
Of the 93 papers, initially selected as having been stopped after an interim analysis, 28 (30%) were stopped early for benefit, 28 (30%) for futility, and 4 (4%) for harm.
As described above, 25 of 28 papers were actually included in the analysis (see Figure 1). All 25 were RCTs, on a variety of different cancers (Table 1). In 16 trials, the control arm used an active comparator and in four used a placebo, while in five no treatment was given. In no case was information provided about trial design in terms of superiority, non-inferiority, or equivalence.
|
More than half of the selected trials (56%) were published in the last 3 years (2005–2007), 11 of them (79%) were used to support an application for marketing authorisation at the EMEA and FDA (Table 2).
|
The evaluation of efficacy was protocol planned through time-related primary end points, >40% of them overall survival (10 of 23, as information was unavailable in two cases). In two cases, publications lacked a clear definition of the primary end points, for both the final and the interim analyses. In 95% of studies (22 of 23), at the interim analysis, efficacy was evaluated using the same end point as planned for the final analysis. There was no DSMC in 24% (6 of 25) of the studies.
All RCTs reported consequences after the interim analysis. Those fell into three groups: cross-over to the treatment group, stopping enrolment, and disclosure of results (Table 2).
The criteria for planning an interim analysis were based either on a cut-off date (3 of 24, as in one case information was missing) or on the number of observed events (12 of 24) or on a preset number of patients enrolled (9 of 24). In 15 RCTs, interim analysis was done when 
50% of the planned sample size for final efficacy analysis was reached. Five, however, reported an interim analysis conducted on a sample 
43% of that planned for the final analysis. This information was not assessable for the remaining five RCTs.
The full sample size initially planned was 8000 patients/events across all trials retrieved. As a consequence of early stopping after the interim analysis, 3300 patients/events across all studies were spared.
The mean study duration was 30 months (range 12–64 months). The median time lag between the end of enrolment (which coincides approximately with study termination) and the date of publication of the results in peer-reviewed journals was 22 months (range 3 months to 15 years).
|
discussion and conclusions |
|---|
|
TopAbstractbackgroundobjectivematerials and methodsresultsdiscussion and conclusionsAcknowledgementsReferences |
|---|
Truncated RCTs reported as having been stopped early for benefit are becoming more frequent. Our findings highlight a consistent increase (>50%) in prematurely stopped trials in oncology during the last 3 years in comparison to whole period analysed (1997–2007).
Ethical reasons also play a role in the decision to stop a trial, since there is a responsibility to minimise the number of people given an unsafe, ineffective, or clearly inferior treatment. On the other hand, an interim analysis may also have drawbacks, since stopping trials early for apparent benefit will systematically overestimate treatment effects [32].
The studies analysed were formally well designed; all were randomised, controlled, on the basis of robust end points, and with a large sample size. Though criticism of the poor quality of oncological trials seems out of place, unfortunately early termination raises new important concerns. Our findings lead to a new awareness: oncological trials are now formally better designed than in the past, but they are too often stopped prematurely. This may cause harm resulting from unreliable findings prematurely translated into clinical practice. More than 78% of the RCTs published in the last 3 years with an interim analysis ending the trial were used for registration purposes. This suggests a commercial component in stopping trials prematurely.
Regarding the methodology used to conduct the interim analyses, sample sizes used to obtain the interim efficacy results varied widely. Substantial concern is raised by five studies which enrolled <40% of the sample planned for final analysis. It is obvious that the risk of overestimating treatment effects increases markedly when the sample is small. Therefore, it is very important to insist that a large number of events must occur before investigators or DSMCs examine interim data, although that cannot guarantee data reliability in any case. In addition, the heterogeneity in sample sizes indicates that these committees enjoy ample discretion in advising or deciding whether to stop a clinical trial early for benefit.
Statistical simulations have shown that RCTs can overestimate the magnitude of the treatment effect depending on the timing of the decision to stop (i.e. the fraction of the total planned sample size or expected number of events) [33]. Furthermore, repeated interim analyses at short intervals raise concern about data reliability: this strategy risks looking as though it is seeking the statistical significance necessary to stop a trial. In addition, repeated analyses on the same data pool often lead to statistically significant results only by chance [34, 35].
If a trial is evaluating the long-term efficacy of a treatment of conditions such as cancer, short-term benefits, no matter how significant statistically, may not justify early stopping. Data on disease recurrence and progression, drug resistance, metastasis, or adverse events, all factors that weight heavily in the benefit/risk balance, could easily be missed. An early stop may reduce the likelihood of detecting a difference in overall survival (the only relevant end point in this setting) because of the small sample, the possibility of crossing-over the experimental drugs, and contamination with other treatments.
Interim analysis data should always be evaluated by a DSMC, which should be independent in the sense that the members should have no interests in the study and should not directly participate in it. Although the majority of RCT reports stated there was a DSMC, we believe that its independence should always be reported. Stopping a trial after an interim analysis is often motivated by ethical considerations. The large number of patients spared (40%), as evidenced by our analysis, might support this. However, the relation between sparing patients and saving time and trial costs is also unquestionable and indicates that there is also a market-driven intent. Our findings show that only a very small percentage of trials (4%) were stopped early because of harm, i.e. serious adverse events, which is quite acceptable. Therefore, toxicity does not represent the main factor leading to early termination of trials.
Stopping a trial early does not guarantee that patients will receive the apparently beneficial treatment—assuming one believes they should—if study findings are not immediately publicly disseminated. We found long delays between study termination and published reports (2 years), possibly because of confidentiality concerns in light of the current regulatory process. If the trials had continued for these further 2 years, more efficacy and safety data could have been gathered. In addition, such delays further lengthen the time needed for translating trial findings into general practice.
The study suffers one main limitation: since there is no ‘standard’ for reporting interim analysis methodology in scientific journals, there may have been some heterogeneity in this respect and some information might have been missed, affecting the sensitivity of the analysis. This could be overcome if study protocols were publicly available and details of interim analysis were reported better in peer-reviewed journals, e.g. by adoption of the Consolidated Standards of Reporting Trials statement.
In conclusion, a decision whether to stop a clinical trial before its completion requires a complex of ethical, statistical, and practical considerations, indicating that results of RCTs stopped early for benefit should be viewed with criticism and need to be further confirmed. The main effect of such decisions is mainly to move forward to an earlier-than-ideal point along the drug approval path; this could jeopardise consumers’ health, leading to unsafe and ineffective drugs being marketed and prescribed. Even if well designed, truncated studies should not become routine. We believe that only untruncated trials can provide a full level of evidence which can be translated into clinical practice without further confirmative trials.
|
Acknowledgements |
|---|
|
TopAbstractbackgroundobjectivematerials and methodsresultsdiscussion and conclusionsAcknowledgementsReferences |
|---|
We would like to thank J. D. Baggott for precious editorial assistance.
Disclosure of interest: Authors belong to regulatory and research institutions devoted to patients’ and national health services’ interests. Both are non-profit organisations. Authors did not receive any funds for conducting the present analysis.
Received for publication December 18, 2007. Revision received January 25, 2008. Accepted for publication January 28, 2008.
|
References |
|---|
|
TopAbstractbackgroundobjectivematerials and methodsresultsdiscussion and conclusionsAcknowledgementsReferences |
|---|
1. The European Medicines Agency. Guideline on the evaluation of anticancer medicinal products in man (CPMP/EWP/205/95/Rev.3/Corr.2). (2005) London: European Medicines Agency.
2. Bertele’ V, Banzi R, Capasso F, et al. Haematological anticancer drugs in Europe: any added value at the time of approval? Eur J Clin Pharmacol (2007) 63:713–719.[CrossRef][Medline]
3. Apolone G, Joppi R, Bertele’ V, Garattini S. Ten years of marketing approvals of anticancer drugs in Europe: regulatory policy and guidance documents need to find a balance between different pressures. Br J Cancer (2005) 93:504–509.[CrossRef][Web of Science][Medline]
4. Johnson JR, Williams G, Pazdur R. End points and United States Food and Drug Administration approval of oncology drugs. J Clin Oncol (2003) 21:1404–1411.
5. van Luijn JC, Gribnau FW, Leufkens HG. Availability of comparative trials for the assessment of new medicines in the European Union at the moment of market authorization. Br J Clin Pharmacol (2007) 63:159–162.[CrossRef][Medline]
6. Montori VM, Devereaux PJ, Adhikari NK, et al. Randomized trials stopped early for benefit: a systematic review. JAMA (2005) 294:2203–2209.
7. Facon T, Mary JY, Hulin C, et al. Intergroupe Francophone du Myelome. Melphalan and prednisone plus thalidomide versus melphalan and prednisone alone or reduced-intensity autologous stem cell transplantation in elderly patients with multiple myeloma (IFM 99-06): a randomised trial. Lancet (2007) 370:1209–1218.[CrossRef][Medline]
8. Hudes G, Carducci M, Tomczak P, et al. Global ARCC Trial. Temsirolimus, interferon alfa, or both for advanced renal-cell carcinoma. N Engl J Med (2007) 356:2271–2281.
9. Motzer RJ, Hutson TE, Tomczak P, et al. Sunitinib versus interferon alfa in metastatic renal-cell carcinoma. N Engl J Med (2007) 356:115–124.
10. Escudier B, Eisen T, Stadler WM, et al. TARGET Study Group. Sorafenib in advanced clear-cell renal-cell carcinoma. N Engl J Med (2007) 356:125–134.
11. Sandler A, Gray R, Perry MC, et al. Paclitaxel-carboplatin alone or with bevacizumab for non-small-cell lung cancer. N Engl J Med (2006) 355:2542–2550.
12. Geyer CE, Forster J, Lindquist D, et al. Lapatinib plus capecitabine for HER2-positive advanced breast cancer. N Engl J Med (2006) 355:2733–2743.
13. Demetri GD, van Oosterom AT, Garrett CR, et al. Efficacy and safety of sunitinib in patients with advanced gastrointestinal stromal tumour after failure of imatinib: a randomised controlled trial. Lancet (2006) 368:1329–1338.[Web of Science][Medline]
14. Goldberg RM, Sargent DJ, Morton RF, et al. Randomized controlled trial of reduced-dose bolus fluorouracil plus leucovorin and irinotecan or infused fluorouracil plus leucovorin and oxaliplatin in patients with previously untreated metastatic colorectal cancer: a North American Intergroup Trial. J Clin Oncol (2006) 24:3347–3353.
15. Akasu T, Moriya Y, Ohashi Y, et al. National Surgical Adjuvant Study of Colorectal Cancer. Adjuvant chemotherapy with uracil-tegafur for pathological stage III rectal cancer after mesorectal excision with selective lateral pelvic lymphadenectomy: a multicenter randomized controlled trial. Jpn J Clin Oncol (2006) 36:237–244.
16. Reed NS, Poole CJ, Coleman R, et al. Scottish Gynaecological Cancer Trials Group (SGCTG). A randomised comparison of treosulfan and carboplatin in patients with ovarian cancer: a study by the Scottish Gynaecological Cancer Trials Group (SGCTG). Eur J Cancer (2006) 42:179–185.[CrossRef][Medline]
17. Lehmann J, Franzaring L, Thuroff J, et al. Complete long-term survival data from a trial of adjuvant chemotherapy vs control after radical cystectomy for locally advanced bladder cancer. BJU Int (2006) 97:42–47.[CrossRef][Medline]
18. Romond EH, Perez EA, Bryant J, et al. Trastuzumab plus adjuvant chemotherapy for operable HER2-positive breast cancer. N Engl J Med (2005) 353:1673–1684.
19. Piccart-Gebhart MJ, Procter M, Leyland-Jones B, et al. Herceptin Adjuvant (HERA) Trial Study Team. Trastuzumab after adjuvant chemotherapy in HER2-positive breast cancer. N Engl J Med (2005) 353:1659–1672.
20. Hurwitz HI, Fehrenbacher L, Hainsworth JD, et al. Bevacizumab in combination with fluorouracil and leucovorin: an active regimen for first-line metastatic colorectal cancer. J Clin Oncol (2005) 23:3502–3508.
21. Goss PE, Ingle JN, Martino S, et al. A randomized trial of letrozole in postmenopausal women after five years of tamoxifen therapy for early-stage breast cancer. N Engl J Med (2003) 349:1793–1802.
22. Moore MJ, Hamm J, Dancey J, et al. National Cancer Institute of Canada Clinical Trials Group. Comparison of gemcitabine versus the matrix metalloproteinase inhibitor BAY12-9566 in patients with advanced or metastatic adenocarcinoma of the pancreas: a phase III trial of the National Cancer Institute of Canada Clinical Trials Group. J Clin Oncol (2003) 21:3296–3302.
23. Yang JC, Haworth L, Sherry RM, et al. A randomized trial of bevacizumab, an anti-vascular endothelial growth factor antibody, for metastatic renal cancer. N Engl J Med (2003) 349:427–434.
24. Arpino G, Nair KM, Doval DC, et al. Idoxifene versus tamoxifen: a randomized comparison in postmenopausal patients with metastatic breast cancer. Ann Oncol (2003) 14:233–241.
25. Radford JA, Rohatiner AZ, Ryder WD, et al. ChlVPP/EVA hybrid versus the weekly VAPEC-B regimen for previously untreated Hodgkin's disease. J Clin Oncol (2002) 20:2988–2994.
26. Noda K, Nishiwaki Y, Kawahara M, et al. Japan Clinical Oncology Group. Irinotecan plus cisplatin compared with etoposide plus cisplatin for extensive small-cell lung cancer. N Engl J Med (2002) 346:85–91.
27. Press OW, LeBlanc M, Lichter AS, et al. Phase III randomized intergroup trial of subtotal lymphoid irradiation versus doxorubicin, vinblastine, and subtotal lymphoid irradiation for stage IA to IIA Hodgkin's disease. J Clin Oncol (2001) 19:4238–4244.
28. Frasci G, Lorusso V, Panza N, et al. Gemcitabine plus vinorelbine yields better survival outcome than vinorelbine alone in elderly patients with advanced non-small cell lung cancer. A Southern Italy Cooperative Oncology Group (SICOG) phase III trial. Lung Cancer (2001) 34(Suppl 4):S65–S69.[CrossRef]
29. Comella P, Frasci G, Panza N, et al. Randomized trial comparing cisplatin, gemcitabine, and vinorelbine with either cisplatin and gemcitabine or cisplatin and vinorelbine in advanced non-small-cell lung cancer: interim analysis of a phase III trial of the Southern Italy Cooperative Oncology Group. J Clin Oncol (2000) 18:1451–1457.
30. Lau WY, Leung TW, Ho SK, et al. Adjuvant intra-arterial iodine-131-labelled lipiodol for respectable hepatocellular carcinoma: a prospective randomised trial. Lancet (1999) 353:797–801.[CrossRef][Web of Science][Medline]
31. Kurth K, Tunn U, Ay R, et al. Adjuvant chemotherapy for superficial transitional cell bladder carcinoma: long-term results of a European Organization for Research and Treatment of Cancer randomized trial comparing doxorubicin, ethoglucid and transurethral resection alone. J Urol (1997) 158:378–384.[CrossRef][Medline]
32. Pocock SJ. When (not) to stop a clinical trial for benefit. JAMA (2005) 294:2228–2230.
33. Schulz KF, Grimes DA. Multiplicity in randomised trials II: subgroup and interim analyses. Lancet (2005) 365:1657–1661.[CrossRef][Web of Science][Medline]
34. Grant A. Stopping clinical trials early. BMJ (2004) 329:525–526.
35. Doll R. The role of data monitoring committees. In: Clinical Trials—Duley L, Farrell B, eds. (2001) London: BMJ Publishing Group. 97–104.
CiteULike 
Connotea 
Del.icio.us What's this?
Related articles in Ann Oncol:
- In this issue
Ann Oncol 2008 19: 1211.[Extract] [Full Text]
This article has been cited by other articles:
|
|
 |
|
  E. L. Korn, B. Freidlin, and M. Mooney Stopping or Reporting Early for Positive Results in Randomized Clinical Trials: The National Cancer Institute Cooperative Group Experience From 1990 to 2005 J. Clin. Oncol., April 1, 2009; 27(10): 1712 - 1721. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Garattini and I. Chalmers Patients and the public deserve big changes in evaluation of drugs BMJ, March 31, 2009; 338(mar31_3): b1025 - b1025. [Full Text]
|
|
|
|
 |
|
 P. Bradbury, R. Meyer, J. Pater, D. Tu, L. Seymour, L. Shepherd, and E. Eisenhauer Stopping a trial early in oncology: for patients or for industry? Ann. Onc., February 1, 2009; 20(2): 395 - 396. [Full Text] [PDF]
|
|
|
|
|
|
J. Whitehead and E. Cobo Stopping a trial early in oncology: for patients or for industry? Ann. Onc., August 1, 2008; 19(8): 1512 - 1513. [Full Text] [PDF]
|
|
E-letters:
Read all E-letters
- Stopping a trial early in oncology: a further element of discussion.
- Antonio Jirillo
- Annals of Oncology, 27 May 2008 [Full text]
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||