{'fraction_non_alphanumeric': 0.027409738323948326, 'fraction_numerical': 0.012669758198078834, 'mean_word_length': 5.193076923076923, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Elicitation of prior probability distributions for a proposed Bayesian randomized clinical trial of whole blood for trauma resuscitation

BACKGROUND Whole blood trauma resuscitation is conceptually appealing and increasingly used but lacks evidence. A randomized controlled trial is needed but challenging to design. A Bayesian approach might be more efficient and more interpretable than a conventional frequentist design. We report the results on an elicitation meeting to create prior probability distributions to help develop such a trial. METHODS In‐person expert elicitation meeting, based on Sheffield Elicitation Framework methodology. We used an interactive graphical tool to elicit the quantities of interest (24‐hour mortality and certainty required). Two rounds were conducted, with an intervening discussion of deidentified responses. Individual responses were aggregated into probability distributions. RESULTS Fifteen experts participated. The pooled belief was that the median 24‐hour mortality of trauma patients with hemorrhagic shock treated with component therapy (the current standard of care) was 19% (95% credible interval [CrI], 6%‐45%), and the median 24‐hour mortality of those treated with whole blood, 16% (95% CrI, 5%‐39%). The pooled prior distribution for the relative risk had a median of 0.84 (95% CrI, 0.26‐3.1), indicating that the expert group had a 64% prior belief that whole blood decreases 24‐hour mortality compared to component therapy. CONCLUSIONS Experts had moderately strong beliefs that whole blood reduces the 24‐hour mortality of trauma patients with hemorrhagic shock. These data will assist with the design and planning of a Bayesian trial of whole blood resuscitation, which will help to answer a key question in contemporary transfusion practice.

B leeding is the most common cause of preventable death after injury, and blood transfusion is an essential part of trauma resuscitation. The current standard of care is the balanced administration of red blood cells, plasma, and platelets, 1,2 which essentially attempts to reconstitute whole blood. Use of whole blood from the outset, rather than reconstituting it from its components, is therefore conceptually and logistically attractive.
Whole blood was the resuscitation product of choice until the development of modern blood component separation techniques. 3,4 Compared with component therapy, whole blood offers several potential advantages, including its balanced composition, ease of administration, and longer shelf life compared with thawed plasma and platelets. Recent military experience with fresh whole blood has stimulated renewed interest in its use. Several observational studies [5][6][7][8][9] suggest improved outcomes, but these types of studies are susceptible to confounding and other issues, such as inability to match. 10,11 A single, small, randomized controlled trial has confirmed the feasibility of such a study, but was not powered to detect mortality differences. 12 Despite this paucity of high-quality evidence, more than 30 US trauma centers and a smaller number of emergency medical services 9 have begun to use whole blood. There have been no large-scale multicenter clinical trials testing the effectiveness of whole blood in trauma resuscitation. We have been working on designing such a trial, the Trauma Resuscitation With Group O Whole Blood Or Products (TROOP) trial. In brief, the aims of the TROOP trial will be to evaluate the clinical and cost effectiveness of whole blood for in-hospital trauma resuscitation of patients in hemorrhagic shock, compared to standard component therapy. The primary outcome will be 24-hour mortality, as it is increasingly recognized that the evaluation of hemostatic interventions may benefit from the use of shorter-term mortality outcomes.
Designing such a trial is challenging. Traditional trial designs would demand large numbers of participants. Assuming a 15% 24-hour mortality rate with standard of care (based on the results of the Pragmatic, Randomized Optimal Platelet and Plasma Ratios [PROPPR] trial), 80% power, and an alpha of 5%, demonstrating a 2% reduction in mortality would require approximately 10,000 participants (5000 in each arm). Based on experiences with the PROPPR trial, such a study would be prohibitively expensive, and take between 10 and 15 years to complete. Even a less conservative (or more ambitious) estimate of the benefits of whole blood, say a 4% absolute reduction in 24-hour mortality, would require around 2300 patientsnearly four times the number enrolled in the PROPPR trial.
In addition to sample size considerations, the interpretation of frequentist designs is also difficult. [13][14][15] In contrast, Bayesian trial designs provide direct estimates of the probability of treatment benefit or harm. 16 Bayesian analyses can also incorporate information from prior studies and experts' judgments, providing results that reflect the influence of existing knowledge or assumptions of effect. While a detailed description of the Bayesian statistical framework is beyond the scope of this article, the approach-as applied to a clinical trial-involves the combining of existing (or prior [the term prior probability distribution, often shortened to prior or prior distribution, refers to the mathematical representation of previously available information]) knowledge regarding the effect of the intervention (e.g., in terms of 24-hour mortality) with new data, gathered from the trial itself (likelihood), yielding the posterior probability distribution. [17][18][19][20] Such beliefs may be based on an examination of the published evidence, personal experience, other sources of information, or a combination of the above. The process of quantifying such beliefs into a prior distribution is known as an elicitation.
In this study, we describe an elicitation workshop to elicit expert opinion of the efficacy of whole blood. The elicited prior distributions will be used to plan and model the proposed TROOP trial, comparing whole blood to component therapy in trauma resuscitation.

Design
We used the Sheffield Elicitation Framework methodology, as described by O'Hagan, with some modifications. 21 Whenever possible, we adhered to good practice recommendations for eliciting expert (the term expert is commonly used to denote the participants of the elicitation exercise; in the context of this report, the term refers to both participation in the elicitation, and individuals' subject matter expertise regarding whole blood) opinion 20,22,23 including preparation of the participants for the elicitation workshop, use of an IRB-approved elicitation protocol, provision of feedback to experts, and an opportunity to revise elicited responses. 20 There are a large number of techniques available for eliciting information. [24][25][26] None have been shown to be superior, but the "roulette method" (also called "bins and chips") may be the most intuitive for clinicians. 27

Setting
The elicitation meeting was conducted at the Houston Marriott Airport Hotel, Texas, on May 7, 2019.

Participants
We invited civilian and military clinicians and researchers with experience in whole blood resuscitation from US trauma centers that currently use whole blood. We reasoned that these individuals would have both knowledge of the published evidence for using whole blood and personal experience. Participants were paid travel and accommodation expenses, and a small honorarium. Eleven trauma surgeons, two transfusion medicine specialists, one pediatric intensivist, and one nonclinical scientist, representing 15 trauma centers currently using whole blood, participated. As transfusion practices and the availability of whole blood differ between countries, we restricted the participants to those working in the United States.

Quantities of interest
The quantities of interest chosen to inform the design of the proposed TROOP trial were 1) the 24-hour mortality rate for trauma patients in hemorrhagic shock who receive component therapy (the current standard of care); 2) the 24-hour mortality rate for trauma patients in hemorrhagic shock who receive whole blood; and 3) the probability of benefit experts would need to consider using whole blood for trauma patients in hemorrhagic shock, assuming no observed differences in rates of adverse events or cost (i.e., all other things being equal).

Online elicitation tool
We developed an interactive online graphical tool based on previous work by Mason et al., 25 using R software and the Shiny package (https://mms-ccrebm.shinyapps.io/ TROOPOnline/). 28,29 Users first entered a subjective probability interval to describe a plausible range for the 24-hour mortality rate. Participants were then asked to provide their "best guess" or "most likely" estimate. These quantities were encoded using a beta probability distribution, which is bounded by [0,1], as probabilities are, where the plausible range represents a 95% probability interval and the best estimate of the mode. The interactive tool then displayed a graphical representation of the corresponding beta distribution (highlighting the 95% probability interval) to give realtime feedback to the participants and allow them to change their estimates until the distribution matched their opinion. When satisfied with their estimates, participants proceeded to the next component of the elicitation. Information provided prior to workshop Before the workshop, we provided the participants with an overview of the elicitation process and the concept of subjective probabilities (Fig. S1, available as supporting information in the online version of this paper). We also provided a simple example (based on Grigore et al. 30 ) to familiarize them with the online tool. Finally, we included examples of Bayesian trials and analyses using informative priors. 31,32 Two weeks before the elicitation workshop, we sent an evidence dossier to participants. The dossier consisted of a list of relevant published studies and conference abstracts on whole blood and component therapy (Fig. S2, available as supporting information in the online version of this paper.). We also asked participants to provide us with any other published or unpublished studies or abstracts of which they were aware.

Workshop program
The workshop was split into seven parts: 1. Presentation of background information on the rationale for the planned TROOP trial of whole blood versus component therapy. We did not present evidence relating to component therapy or whole blood resuscitation at this point to avoid bias by "anchoring" the participants. 2. Introduction to Bayesian principles focusing on the distinction between probability under frequentist and Bayesian paradigms. We also emphasized that Bayesian probability represents the subjective level of uncertainty of an event happening and can vary among individuals. 3. Introduction to elicitation. We then detailed the parameters that we wanted the participants to provide: the lower and upper bound and most likely value of 24-hour mortality in patients in hemorrhagic shock treated with either component therapy (standard of care) or whole blood. 4. Elicitation training exercise. As an example, we used the number of squirrels in Central Park, New York. We worked through this example with the participants, who used an online tool similar to the one used for the actual elicitation, to increase familiarity with the process. 5. Elicitation-Round 1. Participants' beliefs for the parameters of interest were elicited using the online elicitation tool. Figure S1, available as supporting information in the online version of this paper, shows a screenshot of the questions posed to participants. We clarified any and all questions about the statistical terms that participants had during the elicitation exercise but refrained from providing any numbers or information pertaining to the parameters of interest. We calculated prior distributions for each participant's elicited beliefs, and then graphed and presented deidentified individual responses. 6. Group discussion. Participants were then encouraged to discuss their choices. We emphasized that the purpose of the discussion was not to come to a consensus but rather to calibrate individual opinions, and to resolve any questions relating to process. 7. Elicitation-Round 2. The second round was designed to allow participants to revise and calibrate their beliefs, and therefore used the same questions as the first. Participants were provided with an individual code, to allow firstand second-round responses to be compared. The results were, once again, presented as deidentified individual responses.  Fig. 2. Level of certainty required that whole blood treatment is beneficial to continue to use whole blood for trauma resuscitation.
The elicitation was supported by three biostatisticians, who explained the concepts and were available throughout the day to answer questions.

Analysis
We used mathematical pooling to aggregate the individual elicited distributions. First, we evaluated participants' elicited priors for the 24-hour mortality rate with use of component therapy. We used equal-weighting linear pooling of the elicited priors to calculate the clinical prior. This method can help mitigate overconfidence and overoptimism, 33 and when there is extensive overlap of all the individual priors, average pooling is an efficient method to derive a group prior. We also calculated implied prior distributions for relative risks from each individual's elicited priors as well as the pooled clinical priors for the two groups.

Permissions and approvals
This study was approved by the University of Alabama at Birmingham (UAB) Institutional Review Board. Participants were advised of this status at the beginning of the elicitation meeting. There were a wide range of responses, in terms of both the "best guess" measure of central tendency and the credible interval. Some of the distributions were skewed. The pooled distributions are shown in Fig. 1B. The pooled belief was that the median 24-hour mortality of trauma patients with hemorrhagic shock treated with component therapy was 22%, with a 95% credible interval (CrI) of 7% to 74%.

A B C D
( Table 1). The individual priors for the 24-hour mortality of trauma patients with hemorrhagic shock treated with whole blood are shown in Fig. 1C. Again, there was a wide range of responses, in terms of both central tendency and degree of confidence. The pooled belief was that the median 24-hour mortality of trauma patients with hemorrhagic shock treated with whole blood was 19%, with a 95% CrI of 5% to 65% (Table 1). Most participants indicated that they would want to be 50% to 90% certain that whole blood treatment was beneficial, assuming no observed differences in rates of adverse events or cost (i.e., all other things being equal) between whole blood and component therapy, to continue to use whole blood for trauma resuscitation (Fig. 2). Following presentation of the results of the first round to the group, participants were then encouraged to discuss their choices before progressing to round 2.

Round 2
All 15 participants submitted responses in this round. Overall, there was considerable convergence of the distributions. The individual priors for the 24-hour mortality of patients with hemorrhagic shock treated with component therapy (the current standard of care) are shown in Fig. 3A, and the pooled responses in Fig. 3B. The pooled belief, after this second round, was that the median 24-hour mortality of trauma patients with hemorrhagic shock treated with component therapy was lower than in Round 1 at 19%, with a 95% CrI of 6% to 45% ( Table 1). The corresponding responses for trauma patients with hemorrhagic shock treated with whole blood are shown in Fig. 3C and the pooled responses in Fig. 3D. The pooled belief after this second round was that the median 24-hour mortality of trauma patients with hemorrhagic shock treated with whole blood was also lower at 16%, with a 95% CrI of 5% to 39% ( Table 1).
The pooled prior distribution for the relative risk had a median of 0.84 and 95% CrI of 0.26 to 3.1 (Fig. 4) with the participants as a group having a 64% prior belief that whole blood will decrease 24-hour mortality compared to component therapy. Fig. 4. Pooled prior distribution for the relative risk, linear (A) and log scale (B).

A B
After this second round, participants indicated that they would want to be 50% to 60% certain that whole blood treatment was beneficial, assuming no observed differences in rates of adverse events or cost (i.e., all other things being equal) between whole blood and component therapy, to continue to use whole blood for trauma resuscitation.
Changes between elicitation rounds Figure S2, available as supporting information in the online version of this paper, shows how participants' responses changed from Round 1 to Round 2. Figure S2A, available as supporting information in the online version of this paper, shows participants' estimates of the measure of central tendency for trauma patients undergoing resuscitation with component therapy. Figure S2B, available as supporting information in the online version of this paper, shows the width of the corresponding credible interval. Figure S2C and D, available as supporting information in the online version of this paper, show the evolution of responses for whole blood. The graphs again demonstrate a convergence of beliefs and increased confidence. They also demonstrate how outliers (very high or low mortality) tend to revise their estimates as a result of the discussion.

DISCUSSION
The ability to incorporate prior knowledge and beliefs into the design of trials is one of the greatest strengths of Bayesian methodology. Prior information is often available, and prior beliefs almost always exist. The inclusion of prior information can help increase trial efficiency and enable trials that would be unfeasible under a frequentist framework.
The results of our elicitation exercise thus provide useful data on experts' current beliefs regarding the use of whole blood for trauma resuscitation. The participants' pooled estimate of 24-hour mortality of patients treated with component therapy (19%) was somewhat higher than the 24-hour mortality observed in patients enrolled in the PROPPR trial (15%), who were treated with a 1:1:1 transfusion strategy (now the standard of care). We did not explore why the participants' estimate was higher than that reported in the literature.
However, the inclusion of informative priors can also affect the internal validity of studies. If high-quality metaanalyses or randomized controlled trials are available, such priors are rarely contentious. Unfortunately, high-quality data often do not exist. Previous studies may have been mostly observational, not generalizable, or have investigated different outcomes to the ones that the trial planners would like to use. All of these caveats apply to the available literature on whole blood. Specifically, a significant proportion of the data on whole blood stems from its use as a freshly collected product to treat military trauma victims. However, clinicians and researchers will often still have beliefs, extrapolated from their reading of the evidence and personal experience, similar to how they practice medicine. Rather than discount this knowledge, elicitation attempts to structure and quantify it, thereby converting that knowledge into a mathematical form that can be used for trial planning and analysis.
As "early adopters," the participants, overall, expectedly had a positive view of the effects of whole blood. The pooled belief was that the use of whole blood resuscitation reduces 24-hour mortality of trauma patients with hemorrhagic shock to 16%. This elicited absolute risk reduction of 3% is relatively small in comparison to the assumed effect sizes that have been used to design similar trials, despite the participant group being self-proclaimed "enthusiasts." It therefore has high face validity and represents a realistic target for a trial. It also highlights the need for an innovative approach, as a frequentist trial design based on a mortality reduction from 19% to 16% would require approximately 5000 patients. We anticipate that a Bayesian design with informative priors, as we have developed here, will require considerably fewer patients, but determination of the precise number will require additional work (which is in progress).
Our expert panel furthermore indicated that a probability of 50% to 60% of whole blood being superior to component therapy would be sufficient for them to continue (or start) to use whole blood (with 50% indicating an equal chance of whole blood or component therapy being better). This probability was, again, relatively low. This makes sense given that the outcome in question was mortality ("even a small probability that whole blood is better than component therapy is good enough for me to use whole blood, especially given the logistical superiority"). It also provides additional justification for the use of a Bayesian trial design, which permits the reporting of actual probabilities of an intervention being superior, rather than dichotomizing results by means of an arbitrarily chosen p value.
Having two elicitation rounds, with an intervening faceto-face discussion, proved useful. The discussion helped to bring out issues, both relating to the methodology of the elicitation, the baseline 24-hour mortality of trauma patients with hemorrhagic shock resuscitated with component therapy, and the perceived benefits of whole blood. This finding correlates with previous studies. 27 The discussions also brought to light a number of other important issues, such as leukoreduction, and the number of units of whole blood available in each site, which we will consider in the design of the trial. These were not formally recorded or analyzed but included questions about whether children should be included, whether whole blood should be leukoreduced, what antibody titer is acceptable, and whether O-positive whole blood can or should be given to women of childbearing age. Similarly, the importance of other outcomesincluding safety, cost effectiveness, and that the administration of whole blood, especially in pressured clinical settings, is much easier than trying to maintain fixed transfusion ratios-was emphasized.
We also discussed a number of methodological issues, including that of participant selection. We had invited clinicians and researchers from trauma centers currently using whole blood, both for their knowledge of the literature, current outcomes in transfused trauma patients, and their experience of using whole blood. As expected, these participants had generally favorable views of whole blood, and the elicited priors should be regarded as being "enthusiastic." When planning and analyzing Bayesian trials, it is often helpful to consider a range of priors, and the TROOP trial will incorporate not only the prior elicited here but also a neutral prior that assumes a 50-50 chance of whole blood decreasing 24-hour mortality and a more skeptical prior, as well as different weightings of these priors, to ensure that the entire spectrum of possibilities has been accounted for.

CONCLUSION
This was, to our knowledge, the first elicitation related to the design of a Bayesian trauma trial. None of the participants had previously participated in an elicitation meeting, and most had had very little exposure to Bayesian trial design. Overall, the feedback that we received was positive, and the exercise has generated useful data and interest in innovative trial designs. Experts believe that whole blood resuscitation of trauma patients with hemorrhagic shock reduces 24-hour mortality by 3%. These data will inform the design and planning of the TROOP trial. Figure S1: Screenshot of elicitation questions posed to participants using an online elicitation tool. Figure S2: Evolution of responses from Round 1 to Round 2.