{'fraction_non_alphanumeric': 0.03118279569892473, 'fraction_numerical': 0.005734767025089606, 'mean_word_length': 4.6727642276422765, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Response to Letter from K Thorlund et al

Dear Editor, First of all we would like to thank Dr Thorlund and co-authors for their interest in our recent publication on cumulative meta-analysis [1]. In their letter, they question three statements we made regarding the comparison of trial sequential analysis (TSA) and sequential meta-analysis (SMA). First, Thorlund et al. discuss our comparison of TSA and SMA with respect to early stopping for futility. Whitehead’s triangular test was used in our SMAs. This test allows acceptance of the null hypothesis when there is little chance that a significant clinically relevant difference will be found given the results so far. An O’Brien– Fleming beta-spending function applied to a TSA might have led to the same conclusion as the triangular test in the SMA. In their discussion Wetterslev et al. mention the possibility to stop initiation of new trials when the beneficial gain is anticipated to be clinically not worthwhile [2]. However, neither Wetterslev et al. [2] nor Pogue and Yusuf [3] actually applied the option to stop a cumulative meta-analysis for futility in their examples. Instead, for the vitamin E example (ref. 27 and fig. 4 in [2]) the conclusion of the TSA was that ‘ . . . the meta-analysis was underpowered’, whereas the SMA showed that the futility boundary was almost reached [1]. Second, Thorlund et al. question the preservation of the overall type I error using the triangular test in an SMA by referring to a paper by Anne Whitehead [4]. This paper indeed mentions a decrease in power and an increase of the significance level, but for a fixed effects analysis and when comparing prospective meta-analyses with all studies starting at the same time versus studies starting at 4-monthly intervals. An improvement in power and significance level is noted for a random effects analysis with the between-trial variance 2 estimated at each inspection. With large random effects, however, Whitehead’s simulations indeed showed a slight increase in significance level, but clearly less when the method-of-moments estimate was adjusted for the underestimation of . This underestimation of 2 is the topic of their third criticism. We fully agree with Thorlund et al. that inaccurate estimation, and in particular underestimation, of 2 can affect the outcome and conclusions of an SMA. We introduced a new estimate for , the Paule–Mandel estimate, as a more robust and statistically optimal estimate [1,5]. We are now busy comparing this estimate with the generally used method-of-moments estimate to study their behavior in an SMA. We welcome the discussion on the comparison of TSA and SMA as it will further improve the methodology for cumulative meta-analyses and the estimation of the strength of evidence they contain.