{'fraction_non_alphanumeric': 0.024749163879598662, 'fraction_numerical': 0.01649944258639911, 'mean_word_length': 5.487704918032787, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Development and Validation of a Model to Predict Postdischarge Opioid Use After Cesarean Birth

A model with three predictors readily found in the electronic health record—inpatient opioid use, tobacco use, and depression or anxiety—accurately estimated postdischarge opioid use.

OBJECTIVE: To develop and validate a prediction model for postdischarge opioid use in patients undergoing cesarean birth.

METHODS:
We conducted a prospective cohort study of patients undergoing cesarean birth. Patients were enrolled postoperatively, and they completed pain and opioid use questionnaires 14 days after cesarean birth. Clinical data were abstracted from the electronic health record (EHR). Participants were prescribed 30 tablets of hydrocodone 5 mg-acetaminophen 325 mg at discharge and were queried about postdischarge opioid use. The primary outcome was total morphine milligram equiva-lents used. We constructed three proportional odds predictive models of postdischarge opioid use: a full model with 34 predictors available before hospital discharge, an EHR model that excluded questionnaire data, and a reduced model. The reduced model used forward selection to sequentially add predictors until 90% of the full model performance was achieved. Predictors were ranked a priori based on data from the literature and prior research. Predictive accuracy was estimated using discrimination (concordance index).

RESULTS:
Between 2019 and 2020, 459 participants were enrolled and 279 filled the standardized study prescription. Of the 398 with outcome measurements, participants used a median of eight tablets (interquartile range 1-18 tablets) after discharge, 23.5% used no opioids, and 23.0% used all opioids. Each of the models demonstrated high accuracy predicting postdischarge opioid use (concordance index range 0.74-0.76 for all models). We selected the reduced model as our final model given its similar model performance with the fewest number of predictors, all obtained from the EHR (inpatient opioid use, tobacco use, and depression or anxiety). CONCLUSION: A model with three predictors readily found in the EHR-inpatient opioid use, tobacco use, and depression or anxiety-accurately estimated postdischarge opioid use. This represents an opportunity for individualizing opioid prescriptions after cesarean birth. M ore than half of opioids prescribed after surgery go unused. 1 Excess prescribing generates opioids available for nonmedical use with important public health implications. For years, the opioid epidemic in the United States has paralleled trends in legal opioid prescribing. 2 Most individuals engaging in nonmedical use report using opioids legitimately prescribed in the past for their own care or prescribed to family members or friends. 3 Additionally, larger opioid prescriptions are associated with persistent opioid use. [4][5][6][7] Yet, opioid prescribing recommendations after surgery remain vague, but urge clinicians to prescribe the lowest dose and shortest duration possible. 8 Cesarean birth is the most common major surgical procedure in the United States, 9 with 1.2 million cesarean births performed annually. 10 Opioid prescribing after cesarean birth is problematic as up to 75% of prescribed opioids go unused, 11 and opioids prescribed postoperatively are linked to serious opioid-related adverse events. 12,13 Yet, a consistent minority of patients report using all prescribed opioids and having unmet pain needs. 11,14 Uniform reductions in opioid prescribing risk poor pain management in vulnerable patients with implications for postpartum depression and other adverse outcomes. 15 Although the American College of Obstetricians and Gynecologists recommends that, "Therapy should be individualized based on the patient's condition.," 16 most prescribing after cesarean birth reflects clinician prescribing patterns rather than patient characteristics. 17 Clinicians lack tools and concrete recommendations for efficiently individualizing opioid prescribing.
To address these needs, we collected detailed patient characteristics and opioid use data to develop, validate, and calibrate a prediction model for postdischarge opioid use in patients undergoing cesarean birth.

METHODS
We conducted a prospective cohort study of patients older than age 18 years who were undergoing cesarean delivery between November 15, 2019, and January 15, 2021, at a single academic medical center. Patients who did not speak English or Spanish, had diagnoses of substance use disorder or chronic pain, experienced major birth-related complications, or tested positive for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection were excluded. All procedures were approved by the Vanderbilt Institutional Review Board, and informed written consent was obtained before collecting any survey or patient data. The study followed TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) guidelines for the transparent reporting of a multivariable prediction models for individual prognosis or diagnosis. 18 Patients were enrolled postoperatively and administered three questionnaires during inpatient stay: a questionnaire assessing inpatient pain satisfaction and medication use, the ORT (Opioid Risk Tool), and the ERS (Euphoric Response Subscale). The ORT is a five-question screening tool to assess risk for opioid abuse among individuals using opioids. 19 The ERS retrospectively assesses subjective responses to first ever use of opioids. Higher ORT and ERS scores are associated with lower opioid effect, less sedation, and more euphoria with opioid use. 20 Demographic variables not reliably available from the electronic health record (EHR) were also queried, including highest education level attained and detailed tobacco use history. We asked patients about a personal history of depression or anxiety and antidepressant use, and assessed the availability of these variables in the EHR. Race and ethnicity were selfidentified for enrolled participants, and these data were collected given prior research highlighting potential disparities in inpatient opioid dispensing. 21 A broad range of data were collected to identify known and novel predictors of discharge opioid use.
Standardized prescriptions at discharge included 30 tablets of ibuprofen 600 mg and 30 tablets of hydrocodone 5mg-acetaminophen 325 mg (150 morphine milligram equivalents). The hydrocodone prescription was selected to comply with state laws regulating opioid prescribing and provide enough opioid to observe a range of use frequencies. 11 The pharmacy delivered all study medications directly to participants' rooms. Starting 14 days after cesarean birth, participants were contacted to complete additional measures. They were asked to locate their opioid container, count the number of leftover tablets, and record how many were used. If participants stated they were still taking opioids, they were recontacted every 7 days until use stopped. For participants who could not be contacted, there were at least three contact attempts performed before they were deemed lost to follow-up. We accessed the Tennessee Controlled Substance Medication Database for all participants to assess whether additional opioid prescriptions were filled during the postpartum period and to confirm dispending of opioids prescribed at hospital discharge. Our prior research demonstrated high correlation between patient-reported opioid use and real-time electronic medication caps in this population. 22 The primary outcome in our prognostic model was the amount of the opioid prescribed at discharge used until 6 weeks postpartum expressed as morphine milligram equivalents. For our primary model we examined participants who were confirmed via CMSD to have received only the prescribed study opioid (30 tablets of hydrocodone 5mg-acetaminophen 325 mg or 150 morphine milligram equivalents). Participants receiving nonstudy opioids from their clinicians based on review of the Controlled Substance Medication Database were examined in a planned separate secondary analysis.
We identified predictor variables a priori informed by prior research, clinical expertise, and literature review. 11,23 All predictors were available before hospital discharge either as data collected from the EHR or patient-reported data from inpatient questionnaires. This resulted in a total pool of 34 predictors, which were ranked by hypothesized strength of association with the primary outcome. Ranking was informed by prior research and literature review.
To avoid model overfitting, hierarchical cluster analysis using Spearman's rho as a similarity measure helped identify correlated predictors. Principal component analysis was used to compute principal components for each variable cluster. These techniques condensed 27 of 34 predictors into seven groups while preserving overall information. The top seven predictors hypothesized to be highly associated with the outcome were not included in the principal component analysis (Table 1).
Because previous work suggested that inpatient opioid use was highly associated with postdischarge opioid use, particular attention was given to this predictor. Specifically, we considered two methods to characterize inpatient opioids administration: 1) total morphine milligram equivalents used per hour of inpatient stay, and 2) total morphine milligram equivalents used in the 24 hours preceding discharge. Additionally, given concerns about underreporting depression or anxiety in the EHR, we asked women about personal history of anxiety or depression or antidepressant use to compare information obtained from the EHR only (depression or anxiety, EHR only) compared with information from all sources including patient-reported outcomes (depression or anxiety, all sources). We also explored the effect of the participants' bedside nurses on opioid use by asking participants whether, in general, they received opioids when offered by their nurse, when they initiated a medication request, or both. Using opioids primarily because they were offered was included as a binary predictor in the model (bedside nurse).
We constructed our primary model using all predictors, with seven principal component groups and seven individual predictors (full model). We then evaluated which measures of inpatient opioid use and depression improved model performance the most, ultimately selecting prior 24-hour opioid use and depression or anxiety from all sources for subsequent models (Appendix 1, available online at http://links. lww.com/AOG/C681). The second model (EHR model) used 17 predictors, three of which were included as principal components, that are routinely available within the EHR (Table 1). For our third model (reduced model), we used forward selection to add predictors in prespecified ranked order until the model's performance reached 90% of the Likelihood Ratio x 2 achieved for the full model.
Our prognostic model for outpatient opioid use was created using a semiparametric proportional odds ordinal logistic model, which allows for arbitrary outcome distributions. 24 Restricted cubic splines were used for continuous predictors to allow for nonlinear associations. Partial effects plots and nomograms were drawn. We calculated and graphically displayed the fraction of explainable outcome contributed by each predictor based on their partial x 2 values. The sample size was selected to accommodate up to 18 degrees of freedom (df) based on the formula 153df, or 270 participants. Therefore, we were adequately powered using only the cohort of women who received standard prescriptions. Predictors and estimated degrees of freedom were selected a priori.
We compared models using assessments of information (likelihood ratio x 2 statistic), complexitypenalized information (Akaike information criterion), and predictive accuracy (discrimination and calibration). Specifically, discrimination (ability to differentiate which participants needed which total opioid dose) was assessed using Somers' D xy rank correlation (D xy ). Somers' D xy measures the strength of association between ordinal dependent and independent variables and is related to the popular concordance index (c-index) by the equation C 5 D xy þ1 2 . When D xy or C51, the predictions are perfectly discriminating. Both C and D xy are expected to be lower for ordinal continuous outcomes than for binary outcomes. Finally, we used the van Houwelingen and le Cessie heuristic shrinkage factor to quantify the amount of overfitting present and estimate the likelihood that the model will reliably predict new observations in another population. A shrinkage factor greater than 0.9 is considered adequate. We then examined the potential performance of our models in future populations by using internal bootstrap validation with 500 replications with replacement. Analyses were conducted by using Stata 15.1 and R 3.6.2.

RESULTS
Four hundred fifty-nine of 552 (83.2%) eligible patients enrolled in the study, of whom 398 (86.7%) completed the primary outcome, self-reported opioid use (in morphine milligram equivalents). Age, insurance status, and EHR-identified race or ethnicity were not different for patients who enrolled in the study and did not enroll in the study. Eleven patients were excluded owing to complications they experienced after enrollment. After accessing the Controlled Substance Medication Database for the remaining 387 participants, 108 (28.0%) participants were noted to have filled a discharge prescription other than the exact study prescription. This occurred primarily because their clinicians did not realize patients were participating in a study and wrote separate prescriptions. The 279 participants who received and filled the standard study prescription were used for the primary model and analysis (Appendix 2, available online at http://links.lww.com/AOG/C681). On average, participants who received nonstudy prescriptions (either different number of tablets or oxycodone, or both) received less morphine milligram equivalents (90 [60-90] morphine milligram equivalents) than the study prescription of 150 morphine milligram equivalents. These patients had shorter lengths of stay, which may have contributed to not receiving the study prescription. They also reported higher inpatient pain scores but used less opioid per hour of inpatient admission. Other similar demographic and clinical characteristics were similar (Table 2).
Among the entire population (N5387), participants spent a median 2.8 days in the hospital and used the median equivalent of seven tablets of hydrocodone 5 mg-acetaminophen 325 mg during their hospital stay (35 morphine milligram equivalents). After discharge, they used a median of eight tablets or 40 morphine milligram equivalents (interquartile range 1-18 tablets) for a median of 8 days (interquartile range 3-11 days). A majority (53%, 207/387) used some opioid, 23.5% (91/387) used no opioid, and 23.0% (89/387) used all opioids. Using all opioids was more common among participants who received nonstudy prescriptions with a lower median total morphine milligram equivalents than the study prescription (49.2% vs 13.5%).
The EHR (problem list, history, or medication list) identified 17.1% (66/387) of participants as having depression or anxiety; however, patientreported information ("have you ever been diagnosed with or taken medication for anxiety or depression") almost doubled the frequency of a depression or anxiety diagnosis in our population (133/387, 34.4%).
Adjusted odds ratios for each predictor are presented in Table 1 and the proportion of explainable outcome variability is presented in Figure 1. Partial effects plots are presented in Appendix 3, available online at http://links.lww.com/AOG/ C681. Adjusted odds ratios could not be estimated individually for predictors that were components of principal components but instead are presented for the principal component group. Of 34 predictors, four were associated with discharge opioid use: inpatient opioid morphine milligram equivalents used, depression or anxiety, tobacco use, and the bedside nurse. We then explored different measures of inpatient opioid use and depression or anxiety by comparing four full models with the combinations of inpatient morphine milligram equivalents per hour compared with inpatient opioid morphine milligram equivalents used in the 24 hours before discharge, and depression or anxiety from the EHR only compared with depression or anxiety from all sources. We selected inpatient opioid morphine milligram equivalents used in the 24 hours before discharge and depression or anxiety from all sources owing to better model performance (Appendix 1, http://links.lww.com/AOG/C681). Tobacco use and the bedside nurse were significant predictors of outpatient opioid use regardless of which inpatient opioid use and depression or anxiety predictor was selected. Combined, these four predictors accounted for 67.2% of the explainable outcome variation in the full model, and inpatient opioid use alone accounted for 56.4% of the explainable variation. The EHR model identified 18 predictors obtained only from the EHR. The reduced model required only three predictors-inpatient opioid use, depression or anxiety, tobacco use-to reach the prespecified model performance (90% of the full model).
Each of the models demonstrated good predictive accuracy to estimate postdischarge opioid use (Somers' D xy 0.46-0.52 and C-index 0.74-0.76 for all models, Table 3). The internal bootstrap validation results were comparable with the primary models (Bootstrap Dxy 0.47-0.48). We selected the reduced model as our final model given its similar model performance with the fewest number of predictors all obtained from the EHR. Application of the van Houwelingen and le Cessie heuristic shrinkage factor suggested that this model would perform 3% worse when applied to a new population (estimate 0.97). A nomogram based on the final reduced model estimates with these three predictors was constructed and is presented in Figure 2 and a web-based calculator that estimates the morphine milligram equivalents used after hospital discharge can be found at https://vumc-chp-halvorson. shinyapps.io/OpioidUseAfterCes_app/. In a secondary analysis, we tested model performance among 108 participants who filled nonstudy prescriptions. Model performance was consistent despite variable opioid prescribing (Table 3).

DISCUSSION
Using prospective data collected from patients undergoing cesarean birth, we developed and validated three novel prognostic models to estimate outpatient opioid use postdischarge. Of these models, a simple model with three predictors that can be obtained from the EHR-inpatient opioid use, tobacco use, and depression or anxiety-performed as well as more complex models. Although predictive models should be replicated in other samples, our model has potential to provide clinicians individualized guidance on discharge opioid prescribing and ultimately reduce excess opioids available for misuse and diversion.
Predictive modeling has been used widely to predict a variety of obstetric outcomes (ie, postpartum hemorrhage, vaginal birth after cesarean) 25,26 and to predict development of opioid use disorder in adults prescribed opioids. 27,28 To the best of our knowledge, this is the first study to systematically build and vali-date a model that predicts the quantity of outpatient opioids used after discharge by patients who underwent cesarean birth (PubMed search query: ["Forecasting"{Mesh} OR forecasting{tiab} OR predictive  Multiple studies have examined associations between patient or surgical characteristics and postdischarge opioid use. [29][30][31][32][33] Our study is consistent with previous observed associations. Studies from the surgical literature consistently find associations between inpatient and outpatient opioid use, but findings related to tobacco use and depression or anxiety are mixed and largely depend on whether these data were collected. 29 One limitation of examining opioid use in the surgical population is anticipated variation in postoperative pain directly related to differences in types of surgery (ie, orthopedic vs urologic) and approach (open vs laparoscopic). Studying cesarean birth has the advantage of being common, with minimal variation in surgical technique.
In the population who underwent cesarean birth, Bateman et al found no association between tobacco history or antidepressant use and outpatient opioid use, although variability in how tobacco and depres-sion variables were defined could explain these differences. Specifically, current tobacco use in pregnancy may not have been included in their definition, and they captured only one parameter of depression, current antidepressant use. 14 In our study, 15% of participants taking antidepressant medications did not report having depression or anxiety suggesting underreporting of depression. Notably, our model confirms prior studies in the population who underwent cesarean birth that generally have not demonstrated associations between outpatient opioid use and surgical characteristics (duration of surgery, addition of tubal ligation, number of prior cesarean births), obstetric events (labor preceding cesarean birth, indication for cesarean birth), or anesthetic type (regional vs general). 11,14,34 Our study provides a means for individualized opioid prescribing after cesarean birth, the most common major surgery in the United States. The finding that inpatient opioid use is the major predictor of outpatient opioid use aligns with existing literature and clinical experience that inpatient use patterns can be a template for outpatient behaviors. Although inpatient opioid use was the dominant predictor, still,  In this context, decision support tools in the EHR could facilitate prescribing but should be designed using methods that are sensitive to the needs of endusers.
The finding that primarily patient, not surgical, characteristics predict outpatient opioid use deserves further attention especially given prior work demonstrating associations between persistent postsurgical opioid use and tobacco use and depression. 5,35 Both nicotine and depression may modulate the endogenous opioid system in ways that decrease pain tolerance and increase need for analgesic medication. 36,37 Consideration of potential mechanisms underlying predictive effects may be valuable in future work.
Another consideration is the known association between increased opioid prescribing and increased opioid use. 12,14,38,39 In our study, 28% of participants received nonstudy prescriptions and were excluded from the original model owing to concerns for this effect. Although our model appears to perform well when applied to the population who received nonstudy prescriptions (lower total morphine milligram equivalents), it is possible that our point estimates could change if we had selected a different quantity of opioid to dispense for the study. This could represent an avenue for further research.
Major strengths of our study include systematic collection of predictor and outcome variables and a robust approach to predictive modeling. We ranked this comprehensive list of predictors a priori by hypothesized strength of association with the primary outcome. This ranking permitted us to consider different methods to capture data about top predictors and identify optimal characterization for model performance. This selection process combined with modern data-reduction techniques decreases the likelihood of overfitting the model and underperforming in a new population. Our model selection deserves highlighting as well. Although opioid use is a continuous outcome, its distribution is not normal. Prior studies have used linear regression to model opioid use 29,30,40 ; however, this model violates the assumption that residuals are normally distributed. In contrast, we employed the proportional odds model, which provides greater flexibility for nonlinear relationships and better predictive capacity under these conditions. 24,41 Finally, despite excluding 28% of participants who received nonstudy prescription from our initial model, our sample size remained adequately powered to construct the model presented.
There are several important considerations with regards to how our three main predictors were collected. We assumed that patients used opioids during hospitalization as needed; however, this does not account for potential contributions of bedside nurses to inpatient opioid use. 42 In our study, almost 40% of women reported using opioids because they were offered by the nurse. Given the association Fig. 2. Nomogram for estimating opioid use after cesarean delivery. To use the nomogram, assign points for each of the three variables by drawing a vertical line from the response to the first row labeled Points. Add up the total points and note this on the line labeled Total Points. Draw a vertical line from this position to the outcome measures below. For example, a patient who uses 10 morphine milligram equivalents (MME) in the prior 24 hours (30 points) has depression (10 points) and is not a tobacco user (0 points) would receive a total of 40 points. Based on a total of 40 points, the patient would be estimated to use a mean of 52 MME and a median of 37 MME after discharge. Their 90th percentile of use would be 140 MME. The probability of using more than 5 MME is more than 80%, using more than 40 MME is 52%, and using more than 95 MME is 20%.
between inpatient and outpatient opioid use, we acknowledge that variation in inpatient nursing practice could ultimately affect our estimates for outpatient opioid use. We also acknowledge that patientreported opioid use could be inaccurate; however, our prior study in this population found high correlation between patient-reported opioid use and data gathered from real-time electronic medication caps that record when a pill bottle is accessed. 21 Finally, including patient-reported information about depression or anxiety increased ascertainment of this diagnosis compared with EHR documentation alone and improved model performance slightly. Future implementation of our model into clinical practice should emphasize robust ascertainment of depression or anxiety history at patient intake and during pregnancy.
We developed three prognostic models that accurately predict outpatient opioid use after discharge in patients who underwent cesarean birth. A simple model with inputs derived exclusively from the EHR performed similarly to more complex models. Our study suggests that opioid use can be predicted at hospital discharge, paving the way forward for individualized opioid prescribing and a reduction in unused opioids.