{'fraction_non_alphanumeric': 0.02685851318944844, 'fraction_numerical': 0.007194244604316547, 'mean_word_length': 5.197622585438336, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Stopping trials for efficacy: an almost unbiased view

There have been many important technical developments in the evolution of the randomized clinical trial, but arguably few are more important than the adaptations of Wald’s seminal work on sequential analysis [1], a methodology that allowed continuous monitoring of incoming data to make a conclusion at the earliest possible time, while controlling the overall error rate. Armitage published the first book showing how this method could be adapted for clinical experiments [2]. However, even before Armitage’s work appeared, the inimitable Bill Silverman, a neonatologist disturbed by a study of his that he retrospectively felt went on far longer than merited, consulted Wald’s colleagues at Columbia and applied sequential procedures to all his subsequent trials on premature infants [3]. Interim analysis allows the investigator to walk more securely the ethical tightrope that is crossed in every clinical trial, between learning too little and too much; subjecting participants to an inferior therapy long after that inferiority is evident. While the use of statistical monitoring and data monitoring committees (DMCs) began in the 1960s, Pocock’s introduction of ‘group sequential’ methodology in 1977 opened the door for this philosophy and technology to be employed widely in large scale studies [4]. Since then, innumerable studies have implemented some version of this approach. Many trials have been halted in mid-stream on the basis of such boundaries, sometimes to great fanfare. However, this new era of widespread interim monitoring has brought a new set of concerns and tensions. Perhaps the most intractable is the problem of balancing information on safety versus efficacy. It is quite common that safety endpoints occur later, and more rarely, than those related to efficacy. A trial stopped ‘early’ for efficacy might promote adoption of a therapy without understanding of its full consequences, either because the trial was too small or duration too short to accrue sufficient safety endpoints, often a problem even if a trial continues to its planned end. In the Women’s Health Initiative, the asynchrony between the cancer and cardiovascular endpoints was anticipated and planned for in creative ways, through DMC exercises and training [5,6]. Another issue faced by those who use monitoring boundaries is that when the trial is stopped because of a high (or low) effect, there is more of a chance that the trial was at a random high (or low), a problem sometimes known as ‘conditional bias’ [7,8]. Although this issue has long been understood by biostatisticians, it has recently garnered some new attention in the evidence-based medicine community. [9–11] These scientists claim, on the basis of empirical examination of ‘early stopped’ studies, that this bias is substantial, with the potential to distort estimates of the risk-benefit balance and meta-analytic estimates of effect. They also contend that the stopping of a study early is sometimes portrayed dramatically, making it difficult to mount subsequent studies on the same topic because of an unfounded belief that the effect is both larger and more certain than it is. These concerns were echoed in a recent wide-ranging summary of issues related to evidence assessment by the director of UK’s National Institute for Health and Clinical Effectiveness (NICE) [12]. The piece by Freidlin and Korn in this issue [13] represents an attempt to quantify the degree of ‘conditional bias’ under some common scenarios, and compare it to the proper alternative. It adds to previous results [7,14,15], building upon the recognition that comparing treatment estimates from ‘early stopped’ trials to those that continue to their planned end is one of apples to oranges. Trials stopped early for efficacy are by definition statistically significant, whereas results from a trial stopped because a target size was achieved can fall anywhere. If one compares statistically significant results to all possible results, the point estimates from the significant studies will be higher, a well-known fact that is the basis of publication bias [16].