{'fraction_non_alphanumeric': 0.021122112211221122, 'fraction_numerical': 0.005115511551155115, 'mean_word_length': 5.580890336590662, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The Challenges of Generating Evidence to Support Precision Medicine.

Major recent advances in science and technology now enable new kinds of measurements that can characterize an individual’s genes, environment, and behaviors in ways not previously possible. For example, high-throughput genomic analysis and mobile technology can provide large volumes of information that is potentially relevant to prediction of disease and, more importantly, prediction of response to interventions that might improve health. Using this information to precisely target such interventions to persons who are most likely to benefit and least likely to experience adverse effects (ie, precision medicine) could yield major advances in health care (if affordable and scalable to populations) and have a large effect on population health and even health disparities. The National Institutes of Health1 is making major investments to support this vision. However, generating evidence that specifically supports precision medicine and individualized targeting of interventions may be challenging. We see several fundamental options for generating such evidence. Option 1 is observational research designed to identify modifiers of the effectiveness of interventions received by patients in the course of health care delivery. Option 2 is subgroup analyses and interaction testing in standard randomized clinical trials (RCTs) of intervention effectiveness. Option 3 is dedicated precision medicine RCTs that directly compare targeted vs untargeted intervention approaches. Of these 3 evidence generation strategies, option 1 can and will be undertaken by the Precision Medicine Initiative Cohort Program, but it will always be vulnerable to confounding (especially confounding by indication) and cannot supplant the critical role that RCTs must have in establishing precision medicine effectiveness. Option 3 will provide the most direct evidence of effectiveness, but dedicated RCTs will be expensive and highly specific for a particular targeting and intervention strategy and will not be an efficient means of discovering and developing precision medicine strategies. In contrast, option 2 (RCT subgroup analyses) provides a potentially efficient means of discovering effect modifiers and identifying subgroups of individuals who will respond, or not, to an intervention. Therefore, the science of subgroup analysis and translation into effective targeting of interventions deserves attention and critical evaluation. In this issue of JAMA Internal Medicine, Wallach et al2 provide a sobering assessment of the science of RCT subgroup analysis. Using prior work by the Discontinuation of Randomized Trials (DISCO) and Subgroup Analysis of Trials Is Rarely Easy (SATIRE) study groups that identified trials with subgroup analyses, Wallach et al2 further focused on articles making subgroup claims that were deemed important enough by the articles’ authors to mention in the abstract of the article (index articles), reviewed those articles to assess which subgroup claims are supported by evidence of statistically significant heterogeneity (defined by Wallach et al2 as a “subgroup finding”), and then systematically searched the literature for “corroboration attempts” where a subsequent RCT or metaanalysis attempted to provide independent confirmation of a subgroup finding in one of the index articles. Few articles (39.3% [46 of 117]) with a subgroup claim in the abstract actually reported statistical evidence of heterogeneity, fewer (10.9% [5 of 46]) were subjected to a future corroboration attempt in an independent study sample, and none were actually corroborated by such an attempt (ie, none of the “pure” corroboration attempts demonstrated the same subgroup finding). As we consider the implications of these findings, some limitations in the analysis by Wallach et al2 are worth bearing in mind. First, let us at least briefly acknowledge the potential value that a subgroup analysis might provide even if it is not accompanied by a statistically significant test of an interaction, such as confirming an effect in an important subgroup of a larger trial (eg, proving statin efficacy specifically for elderly patients within a larger statin efficacy trial). Second, with only 5 so-called pure corroboration attempts cataloged and no real attempt to evaluate the statistical precision of such attempts, the lack of corroboration could simply be due to bad luck (ie, sampling error) and inadequate precision in the corroboration attempt rather than true type I error (falsepositive subgroup findings) in the index articles. While it appears that at least 1 of the corroboration attempts was adequately powered to exclude an important interaction (see the eFigure by Wallach et al2), it is not clear if the other studies were large enough to definitively contradict the index article findings. Note also that Wallach et al2 identified some “modified” corroboration attempts in their literature review; although excluded in their main results, it appears that some of these attempts appeared to partially corroborate prior subgroup findings (see the eTable by Wallach et al2). Third, it may be that corroboration attempts are actually undertaken when there is some reason to doubt the finding in the original article (eg, it was one of many hypotheses tested); therefore, a low confirmation rate among actual corroboration attempts does not necessarily justify casting doubt on all subgroup claims. Given these limitations and mitigating factors, we might disagree with the authors’ conclusion that subgroup findings are “often spurious”: the problem is more absence of evidence than evidence of absence. However, the absence of confirmed subgroup heterogeneity of effects seems problematic as we consider the prospects for precision medicine. How do we address this problem? First, we need to be more careful about how we plan, analyze, review, report, and interRelated article The Challenges of Generating Evidence to Support Precision Medicine Invited Commentary