{'fraction_non_alphanumeric': 0.038041066602121906, 'fraction_numerical': 0.021259429585703336, 'mean_word_length': 4.98792270531401, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The Liver Retransplantation Risk Score: a prognostic model for survival after adult liver retransplantation

Summary High‐risk combinations of recipient and graft characteristics are poorly defined for liver retransplantation (reLT) in the current era. We aimed to develop a risk model for survival after reLT using data from the European Liver Transplantation Registry, followed by internal and external validation. From 2006 to 2016, 85 067 liver transplants were recorded, including 5581 reLTs (6.6%). The final model included seven predictors of graft survival: recipient age, model for end‐stage liver disease score, indication for reLT, recipient hospitalization, time between primary liver transplantation and reLT, donor age, and cold ischemia time. By assigning points to each variable in proportion to their hazard ratio, a simplified risk score was created ranging 0–10. Low‐risk (0–3), medium‐risk (4–5), and high‐risk (6–10) groups were identified with significantly different 5‐year survival rates ranging 56.9% (95% CI 52.8–60.7%), 46.3% (95% CI 41.1–51.4%), and 32.1% (95% CI 23.5–41.0%), respectively (P < 0.001). External validation showed that the expected survival rates were closely aligned with the observed mortality probabilities. The Retransplantation Risk Score identifies high‐risk combinations of recipient‐ and graft‐related factors prognostic for long‐term graft survival after reLT. This tool may serve as a guidance for clinical decision‐making on liver acceptance for reLT.


Introduction
Liver transplantation (LT) is currently the only lifesaving treatment for end-stage liver failure. The global success of LT has extended the lives of many patients worldwide, but the increasing population of people living with a functional liver graft after primary LT also represents a growing pool of individuals who are at risk for graft failure, many of whom may need a retransplantation (reLT) [1]. Additionally, organ scarcity has motivated the transplant community to expand the donor pool by using more suboptimal grafts that are at greater risk of early graft failure [2]. Currently, up to 23% of liver grafts fail necessitating reLT [3].
The outcomes of reLT have improved over time but still remain worse than those of primary LT. The 5-year graft survival rate of reLT is estimated to be about 45-55%, whereas survival in primary LT is around 60-70% [3]. Prognostic models for graft failure after reLT have been developed, but these studies were conducted more than a decade ago [4][5][6][7][8]. The landscape of reLT has changed over the years, and therefore, high-risk combinations of donor and recipient factors are poorly defined for reLT in the current era. Weighting between the prospect of success and urgency remains a challenge for clinical physicians, and careful selection of donor organs is particularly important in current times of organ scarcity.
The aim of this study was to construct and validate a prognostic model for graft survival after adult reLT using the European Liver Transplant Registry (ELTR) database. The model should be easily applicable for transplant physicians to provide prognostic guidance and help improve matching donor grafts with recipients.

Study population
Data on all adult recipients (≥18 years old) who underwent at least one reLT between January 1, 2006 and December 31, 2016 were obtained from the ELTR database. A study request was reviewed and approved by the ELTR data committee. The methods and approach used to obtain the data have been described previously [9]. The ELTR prospectively collects LT data from 174 centers in 33 countries and guarantees data quality by an internally developed control quality program and by regular audit monitoring of the contributing centers [9,10].

Statistical analysis
Significant predictors of time to graft loss were identified using Cox proportional hazards regression. Time to graft loss was defined as the period between the first reLT and second reLT or death (non-death-censored). Recipient factors analyzed included age at transplant, sex, height, weight, body mass index (BMI), blood group (O, A, B, or AB), primary liver disease, hospital status (at home or hospitalized (ward or intensive care unit)), date of inscription on the waiting list, Model for End-Stage Liver Disease (MELD) score, and high urgency (yes/no). Pretransplant levels of serum albumin, bilirubin, creatinine, international normalized ratio, and sodium had high frequencies of missing data (>50%) and thus were not used for regression analysis but contributed to calculating missing MELD scores. Donor factors included age at death/donation, sex, height, weight, BMI, blood group, and type of donor (donation after circulatory death or donation after brain death [DBD]). Living donors and domino donations were excluded. Other variables included cold ischemia time (CIT), country of graft procurement (same as transplant-performing country or outside transplantperforming country), and type of liver graft (full-size, reduced, split, or living).
Four key variables had a significant number of missing values (>30%): recipient MELD score (48%), recipient BMI (49%), and CIT (32%). We ensured maximal case inclusion in the multivariable analysis by including the cases with missing values by assigning them to a separate "missing" category. Missing data are presented by displaying the "missing" category for each variable, but risk scores were created only for subjects without missing data on the variables included in the risk score.
Cox regression analysis was performed using a forward stepwise selection methodology to examine the independent association between candidate predictors and the occurrence of graft loss. Continuous variables were included as continuous or categorized variables in the model to explore potential alterations in the area under the receiver operating characteristics (ROC) curve (AUC) for predicting graft loss. Models using continuous or categorized variables were compared using the DeLong test. To simplify use of the model, variables were categorized if this did not meaningfully alter the AUC. Dichotomizations of continuous variables were determined using ROC curve analysis and Youden's J Index to maximize sensitivity and specificity. In addition, categories were dichotomized when this did not meaningfully alter the AUC. Hazard ratios (HRs) from regression analyses were expressed relative to a reference category (HR = 1.00) defined either by the group that was closest to physiological normal, the group estimated to have the lowest associated graft loss rate, or the largest group. Variables with a P-value <0.1 in the univariable analysis were included in the multivariable analysis, and the variables significant at P < 0.05 were retained in the final multivariable model.

Derivation of the reLT risk score
To improve clinical utility of the model, a simple scoring system was derived based on dichotomized predictor variables in the multivariable model. Points were assigned for each covariate by rounding the HRs of the Cox regression model. In order to assess how the risk score predicts probability of graft loss, the AUC for the risk score in predicting graft loss was analyzed as a binary variable.

Validation of the model
An internal bootstrap model validation was performed using 1000 bootstrap resamples. Internal validity of the risk score algorithm was evaluated by examining the bias-corrected AUC as a measure of model discrimination, and the Brier score and Somers' rank correlation as measures of model consistency and calibration. Brier scores closer to a value of 0 were considered representing good calibration, and Somers' rank correlation values greater than 0 with larger absolute values were interpreted as having a stronger position association between the risk score and the probability of the outcome. Bootstrapping was performed on the cohort of subjects without missing data on the variables included in the risk score.
External validation was performed by testing the performance of the risk score in the Organ Procurement and Transplantation Network/United Network for Organ Sharing (UNOS) database. Data was obtained on all reLTs between January 2006 and December 2016 to validate our model. External validation was performed on subjects without missing data on the variables included in the risk score.

Comparison with other risk scores
The model was compared with previously published risk models if all data needed to calculate the risk score were available from our database [4][5][6][7]. Differences were considered statistically significant when corresponding twotailed P values were less than 0.05. Statistical analysis was performed using STATA (Version 15.0; StataCorp LLC, College Station, TX, USA) and SPSS (version 23; IBM Corp., Armonk, NY, USA). Statistical review was performed by biostatisticians SJS and DZ.

Baseline characteristics
In the study period between 2006 and 2016, 85 067 LTs were recorded, including 5581 reLTs (6.6%). Of all reLTs, 5150 (92.3%) were first reLTs. Of these, a second reLT was performed in 399 (7.7%) patients, of which 29 (0.6%) underwent a third, two underwent a fourth, and one patient underwent a fifth reLT. Graft survival was significantly reduced after the second reLT compared with the first reLT (P < 0.01), but there were no differences in outcomes after a second and third reLT (P = 0.27). Overall graft survival after the first reLT was 60.1% after 1 year, 54.2% after 3 years, 48.8% after 5 years, and 37.1% after 7 years. For the purpose of this study, only patients receiving a first reLT were included for further analysis.
The median recipient age at the time of reLT was 53.8 (46.3-60.9) years, with a median MELD score of 24.7 (17.1-33.0). The most common indication for primary LT was because of a hepatic or biliary malignancy (17.9%), followed by alcoholic cirrhosis (16.7%), cholestatic disease, such as primary sclerosing cholangitis or primary biliary cholangitis (16.0%), and viral hepatitis C (HCV; 13.7%; Table S1). The most common indications for reLT were vascular complications, such as hepatic artery or portal vein thrombosis (20.2%) and primary nonfunction (PNF; 19.0%; Fig. 1a). Patients requiring reLT for PNF were shortest on the waiting list, whereas patients requiring reLT for recurrent liver disease had to wait the longest for a donor liver (median of 2 days vs. 84 days, respectively; Fig. 1b). Superior graft survival was observed after reLT for rejection of the first graft, biliary complications, or recurrent liver disease compared with reLT for PNF, recurrent HCV, or postoperative infection (Fig. 1c).

Identification of risk factors for graft failure
By univariable Cox regression, several recipient variables were associated with graft failure after reLT, including age, BMI, MELD score, primary liver disease, indication for reLT, high-urgency status, and hospitalization (Table S2). Donor variables that influenced graft survival were age, BMI, and cause of death. In addition, we found the timing of reLT and CIT to be significantly associated with graft failure. These variables were combined in a forward stepwise multivariable Cox model. The multivariable model identified seven variables as significantly associated with graft failure: recipient age, MELD score, indication for reLT, recipient hospitalization, donor age, time between primary LT and reLT, and CIT.

Derivation of the reLT risk score
The reLT risk score was then derived using HRs of the variables retained in the multivariable model (Table 1). Point scores were assigned to each variable in proportion to the HR for that predictor to generate a 10-point score. Only variables that were statistically significant were assigned points. The weighted scores were associated with a probability of mortality at any time following reLT ranging 27-81% (Fig. 2a)  Patients were then stratified into three categories to obtain low-risk (0-3), medium-risk (4)(5), and high-risk (6-10) thresholds. These cutoffs were chosen to create a maximum separation between the risk groups. Figure 2b shows Kaplan-Meier survival plots stratified by the three risk groups. Patients with a low-risk score (0-3) had 71.8% (95% CI 68.5-74.9%) survival at 1 year, 56.9% (95% CI 52.8-60.7%) survival at 5 years, and 51.4% (95% CI 46.9-55.8%) survival at 7 years after reLT. Patients in the medium-risk group (4)(5)  In Table 2, example combinations of recipient-and graft-related variables are shown to illustrate the impact of selected factors on the reLT risk score. For example, matching a recipient without risk factors to an older donor but with an expected short CIT results in a risk score of 2 with good expected outcomes after reLT. Matching a recipient with multiple risk factors (e.g., high age, MELD score of 32, and hospitalized for PNF 2 days after primary LT) to an older donor with an expected long CIT results in a risk score of 7 with a high risk of graft failure after reLT. It should be noted that only one indication for retransplantation (the one with the highest score) should be used to calculate the score. For example, if a patient has a bacterial infection and PNF, three points should be added to the score, instead of 4.

Validation of the reLT risk score
The time-dependent AUC value was 0.623 (95% CI 0.574-0.653) at 1 year after reLT. The corresponding Brier score was 0.24, and Somers' rank was 0.19.
External validation of the model was performed in the UNOS database including 3767 reLT patients within the study period. The expected survival rates were closely aligned with the observed mortality probabilities ( Table 3). The time-dependent AUC value was 0.613 (95% CI 0.594-0.631).  Predictor variables were determined using multivariable Cox regression analyses using a forward stepwise methodology. Variables with a P-value <0.1 in the univariable analysis were included in the multivariable model. Score points were assigned for each covariate by rounding the hazard ratios.

Comparison with other risk scores
The reLT risk score developed and validated in the current study provides a significantly better separation between risk categories than that previously published models. A comparison with the risk score by Hong et al. could not be made because several variables were not available in our database [8]. Seven-year Kaplan-Meier survival plots were developed for graft survival according to the three risk groups of the reLT risk score (P < 0.001).
Comparisons between the groups were made using the logrank test. reLT, retransplantation.

Discussion
Retransplantation of the liver remains controversial because of inferior outcomes compared with primary LT, which raises concerns about inappropriate utilization of scarce donor organs. In this study, representing the largest analysis of reLT to date, we present several novel findings. First, based on seven recipient-and graft-related factors, we have developed a simplified and easy-to-use prognostic model for graft survival after reLT. Second, we identified low-risk, medium-risk, and high-risk groups with significant differences in post-reLT survival. Third, this study confirmed that a good outcome after reLT can be achieved after careful selection of recipient and graft factors. Several risk models for reLT have been developed, but all these studies were based on data from more than a decade ago [4,[6][7][8]. Compared with previously published

(b)
Log rank test P = 0.038 AUC: 0.520 Comparisons between the groups were made using the logrank test. AUC, area under the curve; reLT, retransplantation. models by Rosen et al. [4,5], Linhares et al. [7], and Northup et al. [6], our reLT risk score shows better separation of the survival curves between risk categories. Rosen et al. developed a risk model with data from the UNOS registry in 1999, which was validated and refined in 2003 with data from reLTs performed outside the USA (AUC 0.65) [4]. The final model was based on 979 patients and identified recipient serum bilirubin, creatinine, and the time interval to reLT as significant predictors for mortality. In 2006, Linhares et al. [7] developed a risk model based on a smaller cohort of 139 patients (AUC 0.73). Because both studies were performed in the pre-MELD area, the influence of the MELD score on the outcome was not analyzed. More recent studies, including ours, indicate that the MELD score is a strong predictor of survival after reLT, underlining the need for an updated risk model [8,[11][12][13][14]. Furthermore, survival did not seem to be influenced by donor characteristics in the previously mentioned models. In the present study, we found high donor age to be associated with lower survival after reLT, which has also been pointed out by others [8,11]. Around 20-30 years ago, grafts for reLT were most commonly obtained from younger donors, which could have obscured any effect of old donor age. By adding the cause of recipient graft failure to the wellknown donor risk index (DRI), the reLT DRI was developed by Northup et al. in 2007 (AUC not reported) [6]. Noteworthy, significant predictors in our model were not analyzed in their study, such as whether the recipient was hospitalized and the timing of reLT. Both variables were found to be important predictors of outcomes in several other studies as well [7,8,[11][12][13][14][15]. The most recent risk score for survival after reLT was published in 2011 and included 466 reLTs (AUC 0.64) [8]. Similar to our model, early reLT, recipient age, MELD score, and donor age were identified as predictors for graft failure. The cstatistic was comparable with our model (AUC 0.62), but the results were not externally validated, and the cohort was substantially smaller. An updated model based on a more recent cohort also seems desirable as the donor and recipient characteristics have markedly changed in the last decade. For example, donor age is increasing, and fewer patients are retransplanted for HCV with the advent of direct-acting antivirals [16].

(c) (a)
The presence of infection prior to reLT and timing of reLT deserve some further discussion. Although the presence of infection turned out to be a strong predictor of graft failure in the present study, this variable has not been explored in previous risk models for reLT. Graft survival was worst when infection was the indication for reLT compared with all other indications. An infection prior to reLT could have spread systemically during surgery, leading to sepsis in these patients. Besides, chronic infection could have diminished graft function, decreased a patient's physiologic reserve, and may have impacted the severity of abdominal adhesions, which was previously pointed out by others [15,17]. Multidrug-resistant bacteremia in particular has been associated with a worse prognosis in liver transplant recipients [17]. The increased risk of graft failure for patients undergoing reLT between 2 weeks and 3 months after primary LT is also likely to be reflected by often critically ill patients who are at increased risk of infection. Therefore, when few risk factors are present, proceeding to reLT sooner could increase the chances of long-term survival for a patient. Once the time interval between primary LT and reLT exceeds 3 months, the risk of graft failure diminishes [7,8,11,12,18]. One study has even shown survival rates comparable with primary LT when the interval exceeds 1 year [19].
Individual transplant centers may expect a greater demand for reLT because more patients are living with a functional graft after primary LT. In addition, more suboptimal grafts are transplanted, which are at increased risk for postoperative complications and potential reLT. These advances may justify a proactive view of center-based resources and policies for patients requiring reLT [1]. Although a consensus on minimum survival thresholds after LT is difficult to achieve, a 5year survival rate of more than 50% is generally thought to be acceptable [1]. The results of the present study substantiate that good outcomes after reLT can be achieved after careful matching of recipient and graft characteristics. It must be emphasized that the reLT risk score was not developed to decline donor livers based on the score, but rather as a prognostic guidance for transplant physicians. Naturally, the risk score will only be one factor in the final decision to accept which liver for which patient. Balancing between outcome, urgency, utility of donor organs, and fairness will remain necessary until there are extraordinary breakthroughs in providing alternatives to deceased donor livers [20].
The strengths of this study include the large and contemporary patient cohort, the long-term follow-up, and its robustness validated in an external cohort. Moreover, the reLT risk score is simple to facilitate in clinical use, parameters prognostic of survival can be obtained before organ acceptance, and the cutoff values for the three identified risk groups have clinical prognostic significance. Consequently, we believe that our newly developed risk score could be a useful tool for Our study also has some limitations. First, some variables potentially influencing post-reLT survival were not available from the database used. Second, as with other registry studies, a significant amount of missing data were handled in this dataset. Third, we acknowledge that the model performance is modest based on internal bootstrap validation (AUC of 0.62) and external validation in the UNOS database (AUC of 0.61). However, it should be noted that the observed AUCs are comparable with other widely used models in the field of LT. For example, the AUCs of the DRI and ET-DRI are 0.61 and 0.62, respectively [21,22]. It should be noted that the discriminating power of our model seems to be most prominent in the first 6-12 months after liver transplantation. When following the survival curves of the risk groups, they separate early after transplantation but run parallel thereafter. In other words, for those who have survived the first year after reLT, there seems to be a little impact of recipient/graft variables on longer term survival.
In summary, we have developed and validated a novel prognostic model for survival after adult reLT, which is clinically useful and intuitively incorporates recipient-and graft-related characteristics. A good outcome may be achieved in selected reLT recipients. This tool may aid in clinical decision-making on matching donor livers with recipients.
Authorship IMAB: study concept and design, analysis and interpretation of the data, statistical analysis, and writing of the manuscript. MJMW: study concept and design, interpretation of the data, and critical revision of the manuscript. RA: critical revision of the manuscript for intellectual content. WGP: liaison person for the ELTR/ ELITA and critical revision of the manuscript for intellectual content. VK: critical revision of the manuscript for intellectual content. MAH: critical revision of the manuscript for intellectual content. AM: critical revision of the manuscript for intellectual content. JLK: critical revision of the manuscript for intellectual content. AP: critical revision of the manuscript for intellectual content. DFM: critical revision of the manuscript for intellectual content. JP: critical revision of the manuscript for intellectual content. MS: critical revision of the manuscript for intellectual content. DC: critical revision of the manuscript for intellectual content. MA: critical revision of the manuscript for intellectual content. OS: critical revision of the manuscript for intellectual content. SJS: statistical analysis, interpretation of data, and critical revision of the manuscript for intellectual content. DZ: statistical analysis, interpretation of data, and critical revision of the manuscript for intellectual content. RJP: study concept and design, and critical revision of the manuscript for intellectual content. VEdM: study concept and design, interpretation of data, critical revision of the manuscript for intellectual content, and study supervision.

Funding
The authors have declared no funding.

Conflicts of interest
The authors declare no conflict of interest. plant (Ilse Duus Weinreich) are acknowledged for the data cross-check and sharing with the ELTR.

SUPPORTING INFORMATION
Additional supporting information may be found online in the Supporting Information section at the end of the article. Table S1. Baseline characteristics of first liver retransplants (N = 5150; 2006-2016). Table S2. Predictors of graft failure after liver retransplantation by Cox proportional hazards regression before dichotomization (N = 5150; 2006-2016).