{'fraction_non_alphanumeric': 0.026805822576215325, 'fraction_numerical': 0.009118374073056853, 'mean_word_length': 5.343554006968641, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Methodology for computing the burden of disease of adverse events following immunization

Abstract Purpose Composite disease burden measures such as disability‐adjusted life‐years (DALY) have been widely used to quantify the population‐level health impact of disease or injury, but application has been limited for the estimation of the burden of adverse events following immunization. Our objective was to assess the feasibility of adapting the DALY approach for estimating adverse event burden. Methods We developed a practical methodological framework, explicitly describing all steps involved: acquisition of relative or absolute risks and background event incidence rates, selection of disability weights and durations, and computation of the years lived with disability (YLD) measure, with appropriate estimation of uncertainty. We present a worked example, in which YLD is computed for 3 recognized adverse reactions following 3 childhood vaccination types, based on background incidence rates and relative/absolute risks retrieved from the literature. Results YLD provided extra insight into the health impact of an adverse event over presentation of incidence rates only, as severity and duration are additionally incorporated. As well as providing guidance for the deployment of DALY methodology in the context of adverse events associated with vaccination, we also identified where data limitations potentially occur. Conclusions Burden of disease methodology can be applied to estimate the health burden of adverse events following vaccination in a systematic way. As with all burden of disease studies, interpretation of the estimates must consider the quality and accuracy of the data sources contributing to the DALY computation.

For the measurement of health burden, composite burden of disease (BoD) measures-such as the disability-adjusted life-years (DALY) measure-have been developed. 3 To date, there has been limited assessment of the population-level health burden of AEFI using BoD measures. 4,5 Although selected safety aspects of vaccination have a long history of investigation, studies are often limited to rates or risk estimates. A comprehensive estimate of the population-level health burden of events associated with vaccination and the extent of the AEFI burden relative to the disease being prevented through vaccination would be useful contributions to current knowledge. In addition, quantitative estimates of AEFI burden fit well within the scope of benefit-risk methodology 6 for assessment of new or existing vaccines, if the (projected) averted disease burden due to vaccination can also be quantified. 7,8 A quantitative measure of health burden should ideally take into account the frequency of occurrence, severity and duration of illness, the risk of eventual complications, and the risk of mortality, to allow meaningful comparison between heterogeneous conditions and their effects on the full spectrum of health. The utility of summary measures of population health goes beyond that of simple epidemiological indicators such as incidence or mortality rates, as they integrate mortality with morbidity in a single indicator-taking into account both severity and duration of illness/disability-and therefore are suitable for making comparisons between events, vaccine types, age groups, and national or regional populations. The DALY is a commonly used summary measure of population health burden 3,9,10 and is typically applied to compare the relative impact of diseases on a population. Composite burden measures such as the DALY additionally offer a common currency for representing both beneficial (eg, DALYs averted due to a preventive measure such as vaccination) and detrimental (eg, DALYs lost due to AEFI) impacts on health, which is desirable for benefit-risk assessment.
Our principal objective was to assess the feasibility of adapting current disease burden methodology for computing the populationlevel disease burden of vaccination-attributable adverse events. This work was carried out as part of the "Accelerated development of vaccine benefit-risk collaboration in Europe" (ADVANCE) project, launched in 2013, funded by the Innovative Medicines Initiative (http://www.advance-vaccines.eu). The aim of ADVANCE is to help health professionals, regulatory agencies, public health institutions, vaccine manufacturers, and the general public make well-informed and timely decisions on benefits and risks of marketed vaccines by establishing a framework and toolbox to enable rapid delivery of reliable data on vaccine benefits and risks.

| METHODS
We piloted our methodological approach using a selected set of events well recognized as adverse reactions following immunization. Selection was based on the frequency of occurrence and potential severity of the event. Certain events might occur relatively frequently, but although mild can still be responsible for causing disability (ie, vaccination recipient experiences less than full health), or they might occur extremely rarely, but with serious consequences. We estimated the disease burden, in DALYS, for example events falling within both broad categories of AEFI: (1) infrequent, but potentially serious events (idiopathic thrombocytopenic purpura [ITP], anaphylaxis); and (2) relatively frequent, but less serious events (febrile convulsions) (

KEY POINTS
• Burden of disease measures such as the disabilityadjusted life-year (DALY) are frequently employed to quantify population-level disease impact, and effects of interventions.
• DALY methodology is also applicable to adverse events following vaccination, usefully combining event incidence, severity, and duration in a single composite measure.
• By means of a worked example, we describe the computations and data sources required to compute DALYs for adverse events and identify the most probable information gaps.

| Outcome trees/subsequent sequelae
In cases where the occurrence of a certain adverse event can precipitate recurrence of the same event or can increase the risk of severe sequelae later in life, an outcome tree (also known as disease progression pathway) can be specified to incorporate the risk, severity, and duration of subsequent health outcomes. 25

| Mortality
Distinguishing mortality as a causal reaction to vaccination from coincidental death is crucial, given the extreme rarity of vaccination-attributable death. 26

| Over/under-reporting and under-ascertainment
Determination of either background adverse event incidence or direct attribution of the number of events to vaccination is susceptible to over-reporting (misclassification of cases on an electronic health record [EHR] database, or comparable system), under-reporting (misclassification, or failure to report cases seeking health care), and to under-ascertainment ("missing" cases; those do not seek health care). 27 In the presence of the latter two, burden will be under-estimated.
These factors can be problematic for comparison of burden between adverse events, if the extent of under-reporting/ascertainment differs between event types.

| Selection of parameters for the example burden calculation
The single most important outcome required for computing the health burden of AEFI is vaccination-attributable event incidence. By "vaccination-attributable", we do not make the strong assumption that the observed adverse event has a causal relationship with the vaccine itself, but merely that the event is associated with and occurs following administration of the vaccine. By "attributable", we refer to the extent to which the event incidence is associated with vaccination, adjusting for the expected, or background incidence in the population.
The vaccination-attributable event incidence can be measured by various means: for instance, via calculation of incidence rates prior to and post-vaccination by querying an EHR database with linked date(s) of vaccination(s), through primary data collection via cohort or self-controlled case series designs, 28 or from published reports of event incidence. If only background incidence rates (irrespective of vaccination) are available, then vaccination-attributable incidence can be inferred through application of appropriate relative risk estimates and risk window-size to the background incidence (see Appendix Methods 2).
We detail below the choices made for the example burden computation.

| Setting
Our worked example computes the burden of selected adverse events associated with routine vaccinations administered to young children

| Background event incidence rates
For ITP and febrile convulsions, published background incidence rates were located for recent periods (1990-2005 and 1999-2014) from 2 studies using the UK General Practice Research Database (Table 1). For the latter event, data on narrow age groups were reported. We computed vaccination-attributable event incidence and YLD based on incidence rates within these narrow age groups, and then later aggregated to 2 wider age groups (2 to 12 months, and 13 months to <4 years) for reporting purposes. Note that age groups can be fine-tuned to the target ages for vaccination within the routine vaccination schedule, if event incidence rate data are available at a suitable granularity, for instance by month of age.

| Relative risks of vaccination-attributable event
We conducted a pragmatic literature search for relative risks or absolute risks (defined in terms of cases per vaccine dose) for each of the 6 relevant vaccine-event pairs that were classified as known associations in the GRiP reference set. 19 A single effect estimate (RR or absolute risk) for each vaccine-event pair was then chosen for the worked example ( Table 2).
The age groups for which published relative risks or risks per dose were available did not necessarily match the relevant ages within the Anaphylaxis Infrequent/ high Alternative approach usedsee Table 2 Febrile UK vaccination schedule (see Table 2

| Disability weights and durations
We obtained disability weights from the most recent GBD study (GBD 2013), 17 which updates and expands the set of weights elicited for GBD 2010. 29 Proxy weights were selected (by the authors) for 2 of the 3 selected events. Disability durations were retrieved from published sources ( Table 2); for anaphylaxis and febrile convulsions, duration was assumed to correspond to average stay in hospital. 30 occur only rarely. 32 For simplicity, we excluded the potential burden from additional health outcomes from our YLD estimates.
In addition, for all of our selected vaccine-event pairs, death is recognized as extremely rare, at least in industrialized countries (for instance, no deaths following vaccination-associated anaphylaxis were observed in a large USA study 15 ). For simplicity, we excluded mortality and thus YLL from the worked example.
Finally, sensitivity analyses to assess the impact of selected parameters on AEFI burden may be appropriate when there is uncertainty around the choice of an appropriate parameter value. An example is the choice between multiple proxies for a disability weight not present in available sources.

| Computation of YLD
We estimated YLD for 2 age groups 2 to 12 months, and 13 months to <4 years. All estimates were for the year 2005. Point estimates and 95% uncertainty intervals (UIs) were computed using R statistical software. 33 We tabulated YLD per vaccine-event pair and age group, as well as YLD/1 000 000 (ie, adjusting for age-group population size), and additionally computed estimates aggregated over vaccine type and age group. Computational details are provided in Appendix Methods 1, and example R code is provided in Appendix Methods 3.

| RESULTS
Vaccination-attributable incidence rates and YLD for the 3 selected adverse events (ITP, anaphylaxis, and febrile convulsions (Table 1) were computed stratified by age group and vaccine type (Table 3). These results are provided to indicate how AEFI burden estimates can be usefully reported; they should not be regarded as valid estimates, as data and parameters were selected to illustrate the computation only. YLD, as well as YLD per 1 000 000 persons (to facilitate comparisons between age groups, across time. or between populations), are shown in Table 3 (see also Figure 1). The absolute morbidity burden ranged from 0.044 (95% UI: 0.033-0.062) to 1.14 (95% UI: 0.63-2.02).

| DISCUSSION
We have presented methodology for estimating the morbidity burden associated with adverse events following vaccination, which we illustrated in the form of a worked example. Transparency of the computations involved the ability to compare the health burden of different AEFI using a common metric, and the expected ease of deployment beyond the 3 events we investigated are positive attributes of the applied method.
The extra insight provided by computing YLD (over vaccinationattributable incidence only) for a specific vaccine-event pair, is that YLD additionally takes into account the severity and duration (and possible longer-term consequences) of the event. In our worked example, YLD/1 000 000 distinguished the population-level health impact of vaccine-event pairs with very similar attributable incidence rates (eg, DTaP-ITP and DTaP-Anaphylaxis for 2 to 12-month-old infants).
The worked example highlights the major limitation faced, namely data availability. Box 1 provides a summary of the data requirements for burden computation and options for specifying DALY parameters in the case that the preferred data source is unavailable.
Literature search often did not yield relative risk estimates specific for the study population and/or age groups. It was difficult to find large studies (to provide sufficient statistical precision), or studies that were reasonably recent and/or geographically relevant. Conducting a metaanalysis of published (relative) risks for each vaccine-event pair, or estimation of the relative risks in the population might be preferred approaches. The granularity of the relative risks obtained from the literature was variable, with often very broad age groups defined, and for vaccines administered in multiple doses, separate estimates for each dose were not provided. Accordingly, we had to assume identical (relative) risks and risk periods for each dose. In addition, vaccine co-administration-the norm for routine childhood immunization-complicate estimation of vaccination-attributable incidence due to overlapping at-risk periods. Clearly, application of published relative risks to background incidence rates, or use of absolute risks, requires numerous assumptions to be made about generalizability across time, setting, dose, and age group.
For one of our vaccine-event pairs (DTaP-ITP), relative risks were not significantly different from 1.0. Of course, statistical significance of the published RR depends on the presence of a real increased risk, as well as study power, as the number of outcomes is often very small.
Obtaining relative risks-or better, vaccination-attributable event incidence-directly from large EHR databases is a promising approach for improving precision. We stress that decisions regarding the acceptable power of candidate studies (from which parameters such as RR are obtained), and/or if meta-analyses should be conducted, need to be made a priori.
Retrieving the required disability weights from published elicitation studies is a principled approach; however, 2 of our 3 selected events were not included in the most comprehensive and contemporary source available (GBD 2013) 17   Vaccination of infants and children can influence the health state of the parent, for example as anxiety due to the occurrence of an adverse event and/or uncertainty of prognosis. We followed the BoD approach used for diseases and injuries and therefore ascribed burden only to the individual who received the vaccination.
For the current presentation of the BoD methodology, expert consultation was not needed. However, for real-world application of the methods (see Box 1), we recommend that selection of all DALY parameters is guided by clinical knowledge of the adverse events of interest, and/or undergo review by medical experts (safety physicians), as the resulting burden estimates are crucially dependent on the appropriateness, acceptance, and validity of the parameter values chosen. A process of expert consultation is recommended at the data collection stage as well as for interpretation of the findings, for the dual purposes of ensuring scientific accuracy and credibility. 35 DALYs are most usefully evaluated in context, for instance to create a ranking of diseases in terms of burden. For the burden of AEFI, estimates are meaningfully interpreted when they are compared with the burden of the disease(s) the vaccination prevents. If the population-level disease burden averted by a vaccination programme is similarly quantified using the DALY-which importantly allows benefits and risks to be expressed in a common currency-then the current methodology may find useful application within a benefit-risk monitoring platform. For vaccines which confer long-term protection, the unit of time in which disease incidence is measured may need modification to provide a more appropriate benefit-risk assessment, moving for instance from annual incidence to cumulative incidence. Finally, countries that currently use the QALY as basis for decision-making regarding prevention initiatives could consider either to adapt the methodology to the QALY, or to switch to the DALY measure.
In conclusion, BoD methodology can feasibly be applied to estimate the health burden of adverse events following immunization, but interpretation of the findings must consider the quality, appropriateness, and accuracy of all data sources contributing to the DALY computation. Obtaining the required data is the single major barrier towards computing the burden of AEFI.

ETHICS STATEMENT
The authors state that no ethical approval was needed.
BOX 1 Data/parameter requirements, preferred sources, and possible alternatives for computation of the burden of adverse events following immunization using the DALY