{'fraction_non_alphanumeric': 0.033200214579483814, 'fraction_numerical': 0.024438218990284317, 'mean_word_length': 5.251117734724292, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The Effect of Opioid Stewardship Interventions on Key Outcomes: A Systematic Review

Supplemental digital content is available in the text.


Methods for Selecting Patient Safety Practices
Given the current urgency of harms due to opioids, we conducted a systematic review to identify potential patient safety practices for both primary care practice and other settings. Safety practices that were not fully addressed in existing guidelines, systematic reviews, or standards were prioritized.
Opioid stewardship (OS)-similar to antibiotic stewardshipconsists of a range of risk reduction interventions or strategies, often used in combination to prevent adverse consequences from prescription opioids, including misuse, abuse, and OD. 9,10 The range of OS interventions or strategies includes the following, several of which are recommended in the CDC Guideline for Prescribing Opioids for Chronic Pain: • Conduct an individualized assessment of risks and benefits of opioids, and appropriateness of a tapering; tapering slowly to minimize withdrawal symptoms 11 • Avoid co-prescribing opioids and benzodiazepines or other sedative hypnotics (as appropriate) • Use treatment agreements (TAs, also known as controlledsubstance agreements or pain contracts) • Urine drug screening (UDS) • Check Prescription Drug Monitoring Programs (PDMPs) • Pain and functional assessment • Registry of patients with chronic pain or patients on chronic opioid therapy (COT) • Limiting days' supply for acute pain opioid prescriptions • Pill counts to detect aberrant drug-related behavior • Referrals to nonpharmacologic treatment providers (e.g., physical therapy), pain management, behavioral health, or addiction specialists • Risk assessment Besides recommending these specific interventions, most OS initiatives also include implementation strategies to actually change practice; these implementation strategies are not necessarily unique to OS efforts (Table 1). 12,13 The studies included in this review used a range of implementation strategies to change practice, including electronic health record (EHR) tools (e.g., clinical decision support, templates, alerts, integrated PDMPs, auto-populated fields), dashboards for monitoring and/or audit and feedback, provider and staff education and training, academic detailing, committee or task force on opioids, telehealth, and nurse care management.

METHODS
The question of interest for this review was: What is the effect of OS interventions on key process outcomes (e.g., PDMPs, TA, UDS, referrals); intermediate and clinical outcomes (e.g., opioid dosage, opioid prescriptions, OD); and unintended consequences (e.g., change in pain)?

Literature Search Strategy
Two databases (CINAHL and MEDLINE) were searched for articles published from the past 10 years using terms for opioids, the outcomes of interest (opioid abuse, OD, death), and several terms for OS and OS strategies.

Study Selection
The initial search yielded 392 abstracts and an additional 16 studies identified from authors' knowledge of the field, expert recommendation, and reference lists. After removing duplicates, records of 408 studies were screened from which 24 studies were reviewed for full text. Fourteen individual studies and 1 systematic review met the inclusion criteria (Fig. 1).
Studies were included if they evaluated an OS strategy or a multicomponent OS initiative to address potential harms of opioids. Studies that examined only effective pain management approaches were excluded, if they did not concurrently address potential opioid harms. Studies of naloxone (opioid OD reversal drug) prescribing alone were excluded from this review because of their focus on tertiary prevention (OD reversal) versus risk reduction with primary and secondary prevention strategies; no studies included in this review had naloxone prescribing as part of their initiatives.
Studies were included if they used experimental or quasiexperimental designs with pre/post measurement, with or without a control group. Observational studies and qualitative studies without tests of significance or with fewer than 50 patients they were excluded.
Studies were excluded if the outcomes were not relevant to this review (e.g., focused only on clinician outcomes such as knowledge or perceptions), if the article was out of scope, or if the report did not describe an intervention.

RESULTS
The 14 single studies that met the inclusion criteria were characterized in terms of their setting, OS strategies examined, study design, and outcomes (see Evidence Table in

Settings
Ten studies examined [14][15][16][17][18][19][20][21][22][23] OS interventions in primary care settings: 3 of which were in federally qualified health centers (FQHCs) 14,15 or safety net settings 19 and 2 were in Veteran's Administration (VA) clinics. 16,18 One of the 10 studies in primary care settings examined a health systemwide OS initiative, 23 which included primary care practices as well as emergency departments (EDs) and hospitals. Two studies examined OS in EDs, 24,25 1 in a hospital outpatient surgery department, 26 and another in an urgent care setting. 27

Opioid Stewardship Strategies
Most studies examined multicomponent OS interventions, which often consisted of guideline-recommended clinical interventions or care processes (e.g., use UDS, check PDMPs), as well as implementation strategies (e.g., dashboards, audit, and feedback), which are described previously. Descriptions of the various OS initiatives varied in their level of detail. See Table 1 for an indication of the specific components of the OS interventions reflected in the literature included in this review.

Study Designs
Fourteen single studies and 1 systematic review were included in this review. Six of the 14 studies had a control group: 2 were randomized trials, 19,25 3 were interrupted time series with control groups, 17,20,21 and 1 was a 1-way crossover intervention study with patients serving as their own control. The remaining studies included 6 preintervention/postintervention studies without a control or comparison group and 2 observational studies with tests of significance. The postintervention periods ranged in these studies from months to years.
The overall strength of the evidence on OS was ranked low to moderate, with some variation by outcome examined.

Outcomes Examined
The most clinically significant harms of opioids are opioid addiction or opioid use disorder (OUD), OD, and death. Most studies did not examine the effect of OS initiatives on OUD or OD with a few exceptions. 21 Most studies examined the effect of OS interventions on reducing the potential risks of opioids with judicious prescribing and guideline-concordant care (e.g., reduce inappropriate high opioid dosages, avoid co-prescribing opioids and benzodiazepines, use UDS, TAs).
The outcomes are presented by intermediate outcomes, process outcomes and utilization, and OD.

Intermediate Outcomes
Most studies examined intermediate outcomes including opioid prescribing and high opioid dosages and potential misuse.

Opioid Prescribing
Seven studies examined effects of OS on prescribing any amounts of opioids. The evidence is low to moderate that OS efforts decrease numbers of opioid prescriptions, proportion of patients on long-term opioids, or days' supply.
Anderson et al 15 (2015) observed reductions in the proportion of patients on COT after their OS intervention (3.4%-3.1%, P = 0.057). In addition, Weiner et al 23 (2019) found a reduction in the number of unique patients with an opioid prescription each month (−52.6 patients, P < 0.001). 23 Von Korff et al 20 (2016) found a significant decline in the proportion of patients receiving excess opioid days' supplied (from 24.0% to 10.4% among interventions patients on COT and from 20.1% to 14.7% among COT patients for the control practices).
Finally, Hartford et al 26 examined a hospital-based outpatient surgery OS initiative and found that only 78 (45%) of 172 patients filled their opioid prescription in the postintervention group (P < 0.001), with no significant difference in prescription renewals.

High Opioid Dosages
Six studies examined the effect of their OS interventions on opioid dosages, measured as morphine milligram equivalents or MMEs. 16,19,20,22,23,26 Four were in primary care settings, 16,19,20,22 1 health systemwide, 23 and 1 in hospital outpatient surgery. 26 The strength of the evidence for OS initiatives producing significant reductions in opioid dosages was moderate.
Although the OS strategies varied and the postintervention periods ranged from months to years, included studies observed reductions in MMEs with varying magnitudes and measured in various ways. The following is a summary of the findings by the different measures of dosage used in the studies, some of which reported dosage more than 1 way. Von Korff et al 20 (2016) found that mean daily MMEs decreased by 47% compared with control at 30%. Weimer et al 22 reported an average daily dose decreased by 64 mg (95% confidence interval [CI] = 32-96, P < 0.001). In terms of dosage reduction, Liebschutz et al 19 (2017) found that intervention patients had a mean MME 6.6 mg lower than controls (P < 0.001), and intervention patients were more likely than controls to have either a 10% MME dose reduction or opioid treatment discontinuation (adjusted odds ratio = 1.6).
Studies examined high dosage by the proportion of patients on high dosages and observed a range of reductions in patients on high dosages. Von Korff et al 20 (2016) (2019) found a significant decrease in mean MMEs per prescription (−0.4 MME per month, P < 0.001), and prescriptions containing 90 or more MMEs also decreased (−48.1 prescriptions/month, P < 0.001), which may or may not be statistically significant. 23 In the study of the OS initiative in general outpatient surgery, MMEs for prescriptions filled for the intervention group were significantly fewer than for controls. 26

Potential Misuse
Few studies included in this review examined misuse outcomes. One ED study found that the total number of unique controlledsubstance prescribers decreased from 11 to 7 (31% decrease, 95% CI = 23-38), which may or may not equate to fewer prescribers. 24 Another study in primary care found no difference in early refills in their intervention group compared with control. 19

Process Measures and Utilization
The primary outcome targeted by most OS initiatives was to improve utilization of recommended clinical interventions or care processes or "guideline-concordant care." Five studies examined these various process outcomes.

Guideline-Concordant Care
In a randomized trial Liebschutz et al 19 (2017) found that intervention patients were more likely than controls to receive guidelineconcordant care (65.9% versus 37.8%, P < 0.001, adjusted odds ratio = 6.0, 95% CI = 3.6-10.2). Similarly, Jacobs et al 18 found significant improvements in guideline-concordant care after the pharmacist-led intervention in a VA setting.

Urine Drug Testing
Four single studies examined the effect of OS initiatives on the use of annual UDS and observed significant increases in screening rates. 14,16,18,19 In their systematic review, Starrels et al 9 (2010) found low to moderate evidence of the effectiveness of urine drug testing (UDT) for reducing opioid misuse.

Prescription Drug Monitoring Program
One study that examined prescribers' use of the PDMP found significant increases in utilization with opioid prescribing. 18

Treatment Agreements or Informed Consent
Four studies examined the effect of OS initiatives on the proportion of patients on COT with a TA and found significant improvements. 14,16,18,19 In a systematic review, Starrels et al 9 (2010) found that opioid misuse was modestly reduced after TAs (with or without UDT).
Several OS initiatives aimed to increase referral to behavioral health and other specialists. Anderson et al 14

Emergency Department Visits
Two studies examined OS initiatives in EDs and observed significant decreases in ED visits of 34% (decrease from 14 to 4, a 58% decrease, 95% CI = 50-66) 24 and 58% (incident rate ratios = 0.663, P < 0.001, 95% CI = 0.569-0.775). 25 Overdose Two studies 21,23 examined whether their OS initiatives reduced ODs; neither study observed significant reductions. Von Korff et al 21 (2019) found that changes in OD rates among patients did not differ significantly between intervention and control groups with the implementation of 2 different OS initiatives (dose reduction and risk stratification/monitoring). Secondary analyses revealed that OD rates decreased significantly (17% per year) from the dose reduction OS initiative for patients on COT in inter-

DISCUSSION
Most OS interventions are multicomponent, involving both the clinical interventions or care processes and often implementation strategies. The implementation strategies found in this review included education, policies, dashboards, audit and feedback, monitoring and metrics, health information exchange, and EHR tools. The EHR tools included an embedded PDMP, registry, alerts, auto-population features, and templates.
When multicomponent interventions were included in this review, we did not examine the differential effectiveness of different components. None of the included studies used implementation or implementation effectiveness designs 13,28 to afford a systematic evaluation of different implementation strategies' effectiveness. Instead, the researchers of select studies offered reflections and informal observations on facilitators and barriers to implementation of their respective OS initiatives.
Anderson et al 15 (2015) fielded a survey of the participating primary care providers about their opioid dashboard. Respondents found the dashboard helpful for identifying patients on long-term opioids and gaps in services (85%), clinically useful (77%), and easy to use (69%). Electronic health record tools were identified as key facilitators to OS. 16,23 On the other hand, Dorflinger et al 16 (2014) also found EHRs limiting because of the challenges with capturing complementary health approaches (e.g., chiropractic). Weiner et al 23 (2019) reflect on several lessons learned. They found that determining metrics and gaining access to data were critical at the start of an initiative to guide the OS effort. They also experienced conflicting expectations when primary care providers referred patients to pain specialists with the expectation that the pain physicians would prescribe opioids, whereas the specialists would only recommend opioid regimens and provide injections. In addition, although their health system had increased substance use disorder treatment, their outpatient practices perceived that there was inadequate access. Finally, they learned that many of these implementation challenges could be addressed by convening the various stakeholders to resolve the issues. Buy-in and administrative support were identified as key for 2 OS initiatives, as well. 24,25 This systematic review expands the evidence on OS interventions beyond what was known from previous reviews of specific interventions but still points to several gaps and future directions for reducing the potential harms due to opioids. There is a need for more detailed descriptions of the OS initiatives to replicate the interventions in other practices and settings, as well as rigorously synthesize the evidence across studies. In addition, improving the quality of future studies with control groups to account for secular trends, given the attention on the opioid epidemic and changing external environment, policies, regulations, and evidence, is critical. Examining the benefits of different implementation strategies for changing practice in OS efforts and in different settings is also needed. The studies included in this review were not only in primary care settings but also in health systems, in EDs, and an urgent care center; there is still a need to further understand the uniqueness and effectiveness of OS efforts in different settings.
With the evolving epidemic and response to it, it may be important to also examine the effect of co-prescribing naloxone for patients on long-term opioid therapy on outcomes of interest, as well as examiner how best to identify and treat or refer patients using illicit opioids. It should be noted that most OS efforts are aimed at preventing or reducing harms due to opioids with appropriate prescribing and do not examine the unintended consequences.

CONCLUSIONS
This review offers additional insights for the field in responding to the opioid epidemic in the health care system and points to several potential gaps to expand our knowledge of the effective approaches.