{'fraction_non_alphanumeric': 0.028607554923344398, 'fraction_numerical': 0.011748590695959116, 'mean_word_length': 5.15100453661698, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Expanding Latent Tuberculosis Infection Testing and Treatment in Massachusetts Primary Care Clinics via the ECHO Model

Introduction/Objectives: In the US, reactivation of latent tuberculosis infection (LTBI) accounts for 80% of new cases. In 2016, the US Preventive Services Task Force provided a new recommendation that primary care providers (PCPs) should conduct LTBI screening, whereas in the past, LTBI cases were evaluated and treated by specialty providers. This shift in care revealed knowledge gaps surrounding LTBI treatment among PCPs. This study assessed changes in PCPs’ confidence for performing key aspects of LTBI care before and after participation in an LTBI Extension for Community Healthcare Outcomes (ECHO) course. Methods: The ECHO Model™ is an evidence-based telementoring intervention. Participants were primary care team members from clinics throughout Massachusetts who voluntarily enrolled in the ECHO course. In this mixed-methods evaluation, primary outcomes were PCP self-reported confidence changes by pre- and post-course surveys and post-course semi-structured interviews. Results: Twenty PCPs (43% of registered PCPs) attended at least 3 of the 6 sessions and 24 PCPs (31% of registered PCPs) completed at least one survey. Confidence increased in selecting a test (P = .004), interpreting tuberculosis infection test results (P = .03), and selecting a treatment regimen (P = .004). Qualitative interviews with 3 PCPs revealed practice changes including switching to interferon gamma release assays for testing and using rifampin for treatment. Conclusions: Use of the ECHO model to train PCPs in LTBI management is feasible and efficacious. For continuing medical education, ECHO courses can be leveraged to reduce health disparities in settings where PCPs’ lack of familiarity about a treatment topic contributes to poor health outcomes.


Introduction
Tuberculosis (TB) remains a significant cause of global morbidity and mortality. 1,2 In the United States (US), approximately 80% of TB disease is a result of untreated latent TB infection (LTBI). 3 An estimated 13 million individuals in the US have LTBI, up to 10% of whom will progress to infectious TB disease. 4 Unfortunately, while an estimated 72% of at-risk individuals are screened for LTBI, 5 only 20% to 65% of those who begin therapy go on to complete treatment. 6 An increased focus on LTBI screening and treatment is paramount in order to achieve the World Health Organization's goal of TB elimination by 2030. 7 In 2016, the US Preventive Services Task Force recommended screening at-risk individuals for LTBI in the primary care setting with a B rating based on strong evidence. 8 Subsequently, primary care providers (PCPs) were expected to add LTBI testing and treatment to their scope of practicetasks previously left to specialists. Primary care residency training often incorporates minimal education and training related to LTBI, and PCPs are typically unprepared to incorporate testing and treatment into their practices. 9 PCPs often have strong provider-patient relationships, which increases the likelihood of treatment compliance. 10 Identifying strategies to increase LTBI testing and treatment in primary care is a critical component to TB elimination in the US.
One-time educational activities are not sufficient to develop new skills, 11 The Extension for Community Healthcare Outcomes (ECHO) Model™ was developed in New Mexico in 2003 for Hepatitis C treatment and is an effective telementoring intervention designed to build PCPs' capacity to manage complex health conditions and expand patient access to what previously was specialist-supported care. 12 The model has been expanded to address numerous other conditions [13][14][15][16][17] and has demonstrated improvement in provider knowledge and skills as well as patient outcomes. 18 Prior to the start of our primary intervention, other TB-related ECHOs in the US focused primarily on TB disease and were geared toward public health and infectious disease specialists. To our knowledge, this is the first attempt to evaluate an LTBI ECHO model's impact on PCPs' confidence and skills.
The Massachusetts Department of Public Health (MDPH) has integrated infectious disease prevention and control functions across HIV, viral hepatitis, and sexually transmitted infections. 19 Through contracts with hospitals, health centers, and other community-based providers, MDPH supports a range of integrated HIV, viral hepatitis and STI prevention, testing, linkage to treatment, and patient navigation services. In late 2017, this program expanded to include LTBI. MDPH also provides training, education, and capacity building activities to promote high-quality, evidence-based health service delivery. In 2019, the MDPH partnered with the University of Massachusetts Chan Medical School (UMass Chan) Project ECHO Hub to design and implement an LTBI ECHO. The goal of the LTBI ECHO was to enhance the capability and confidence of primary care teams in Massachusetts regarding LTBI management. Here, we describe the implementation of the LTBI ECHO and report the results from our mixed-methods program evaluation. The primary outcomes for this evaluation were to assess the impact of the course on PCPs' confidence in performing various aspects of LTBI care and to explore PCPs' attitudes regarding the LTBI ECHO to improve upon future iterations.

Program Description
The MDPH and UMass Chan Project ECHO hub formed a partnership to bring together LTBI content experts.
The curriculum was informed by a 1-day workshop hosted by the Rutgers Global TB Institute and on discussions with other TB ECHOs (eg, the Washington State Department of Health's ECHO). 20,21 A 6-session curriculum was finalized by the course director (DS) and MDPH personnel and followed through 2 iterations-Cohort 1 and 2, beginning in January 2019 and February 2020, respectively. Each session focused on a critical aspect of LTBI (Table 1). There was no cost to participants but registration was required. Participants received 1 AMA Category 1 Credit for attending each session after completing a post-session evaluation.
Sessions were 1-h in duration from 12:15 to 1:15 pm and occurred monthly on Thursdays. Each session followed the ECHO model: a 15 to 20 min didactic portion followed by a 35 min case presentation/discussion. 22 Sessions were facilitated by 2 PCPs (DS and/or RT). The didactic presentations were presented by members of the Massachusetts Medical Advisory Committee for the Elimination of TB (MACET). Our "Hub Team" included the facilitators and MDPH personnel, including clinical, case management, and public health experts. The course was coordinated by an experienced UMass Chan ECHO coordinator (SF). 23 The LTBI ECHO participants were recruited from clinics under contract with MDPH, community health centers through the Massachusetts League of Community Health Centers, and through the UMass Chan primary care network.

Program Evaluation
This was a quality improvement study in which we used a mixed-methods approach to assess the impact of the LTBI ECHO on PCPs' confidence in performing various aspects of LTBI care and to evaluate the program in order to improve future iterations. Methods included: (1) pre-and post-training quantitative questionnaires; and (2) post-training qualitative key informant interviews. We abided by the SQUIRE 2.0 revised standards for quality improvement reporting excellence.

Ethics
The UMass Chan Institutional Review Board deemed this educational evaluation as exempt.

Data Collection
Quantitative pre-and post-structured questionnaires were administered electronically via Survey Monkey ® to all registered course participants 2 weeks before and within 2 weeks after the conclusion of the 6-session course. Questions focused on feasibility (process, resources, management) and impact (self-reported learning and performance) of implementing an LTBI ECHO program for primary care team members. In the post-structured questionnaire for cohort 1, participants were asked if they would be willing to partake in a qualitative interview. Three participants were randomly selected from the volunteers. Content experts were asked to voluntarily participate in interviews via email. A semi-structured interview guide was created for qualitative interviews (Supplemental Material, S1), which were conducted and recorded via Zoom™.

Data Analysis
Quantitative survey data were imported into Google Inc.'s Jupyter notebook environment, Colab. Summary statistics and tests of significance were calculated using the NumPy python package. Significance testing for changes in responses to identical pre-and post-survey confidence questions was performed using the non-parametric Wilcoxon signed rank test for paired data. Graphical representations of data trends were created using the GraphPad Prism software. For qualitative data, recordings and transcripts were reviewed and data were analyzed using the immersion/crystallization technique. 24 Broad themes and key quotes were summarized into final, meaningful topical segments.

Quantitative
A total of 78 primary care team members registered for either Cohort 1 or Cohort 2, and 45 attended at least one session, an engagement rate of 58% (Supplemental Material, S2). Although many primary care team members were invited to participate in the ECHO course, including administrators, care managers, or community health workers, we limited our analysis to PCP survey respondents given that the survey response rates were low for other cadres. Twenty PCPs (43% of registered PCPs) attended at least 3/6 sessions and 24 PCPs (31% of registered PCPs) completed at least one survey. None of the PCPs who completed surveys in Cohort 1 participated and completed surveys in Cohort 2. Among those PCPs who completed at least one survey, 71% held the degree MD or DO while 29% were nurse practitioners (NPs) or physician assistants (PAs) ( Table 2). Few PCPs (12%) had previously participated in an ECHO program on another topic. Twelve PCPs completed both pre-and post-surveys. The pre-and post-surveys each contained an identical section that asked PCPs to rate their confidence in performing critical LTBI care proficiencies ( Figure 1A). When survey data were matched for each PCP that completed both a pre-and postsurvey, there was a clear trend toward increased confidence for all areas ( Figure 1B). The trends were similar for both  (15) cohorts when they were disaggregated. The Wilcoxon signed rank test was applied to determine whether responses showed significant changes (Table 3, Supplemental Material Figure  S3). Significant increases in reported confidence were observed in 8/10 categories by a 2-tailed test with α = .05.
These categories included "selecting a test to screen for TB infection" (P = .004), "providing education to a patient about TB infection treatment" (P = .01), "selecting an appropriate TB infection treatment regimen" (P = .004), and "reporting TB infection to the Department of Public Health" (P = .01).

Qualitative
Qualitative results. Three ECHO PCPs and 2 ECHO subject matter experts from Cohort 1 were interviewed between August 5-13, 2019. Interviewees were from a variety of specialties including family medicine, pediatrics, and infectious diseases and included 4 physicians and a nurse practitioner. Two of the PCPs attended all sessions while one participant attended 3/6 sessions. All PCP interviewees thought that the course was worthwhile and would participate in future courses if offered. PCPs felt that the material was presented in a way that was accessible to their daily practice, such as the interviewee who stated, "I thought this was great. It felt clinically useful; it felt fun to do. It was not so frequent that it felt like drudgery but it was frequent enough that I had the material in my head. It definitely felt like I was learning every time. I thought it was well facilitated." The themes were summarized as follows: ECHO session timing. Even among the PCPs who attended all sessions, one stated, "getting to ECHO was not easy. . .if I had an afternoon clinic session, I would need to reschedule  patients since my patients start at 1pm. However, this was easier to do since the ECHO was only monthly." Another PCP who attended all of the sessions stated that their health center allowed them to block the last patient of the morning to attend. The one PCP who did not attend all sessions was a global health fellow and due to travel obligations, was out of the country for some of the sessions. Overall, PCPs thought that mid-day was the best time to have the ECHO sessions. "Once a month was about right. If it were more frequent, it would be harder to attend. If more spread out, it would be hard to keep the thread of what you are learning going." The timing for content experts was inconvenient. For one expert, the sessions took place during Grand Rounds. For the other, their office is far away from the hospital -the only place where they could connect to the ECHO. This expert stated that a better time might be around 5 pm.
ECHO session cases. Several interviewees mentioned that case presentations should be more aligned with didactics; however, they also acknowledged the difficulty of coordinating this. One content expert asked if it would be possible to let content experts know more in advance about the case and about possible questions related to the cases.
ECHO session didactics. Several PCPs felt that ECHO content was not equally distributed across the LTBI care cascade as well as across populations. "As a participant, there was a lot of time spent on considerations regarding testing and it took a little while to get to treatment considerations. I felt that in some presentations, speakers were recapitulating things that were said before." Another stated, "For us [pediatricians], there were times the ECHO was more adult-focused." Practice changes potential impact. Several PCPs stated that they made practice changes related to the ECHO. One stated, "We are now shifting to rifampin-everyone was getting 9 months of isoniazid and now I myself started a kid on rifampin for the first time 3 weeks ago. . .It will be so much easier for families to do 4 months of rifampin especially since most of these families are immigrant families.

Discussion
There is a dearth of published evidence regarding LTBI training programs nationally, particularly those providing training for PCPs. Given the shift toward LTBI testing and treatment in primary care, identifying successful training approaches for PCPs is imperative. Our mixed-methods evaluation suggests that an LTBI ECHO may be particularly effective in improving PCPs' confidence. Pre-ECHO surveys identified the proficiencies of "selecting a treatment regimen" and "monitoring a patient on treatment" as areas with the lowest initial confidence. Encouragingly, analysis of paired post-ECHO surveys showed significant increases in PCP-reported confidence for these areas. These findings are consistent with results from other studies evaluating disease-specific ECHO courses. 12,[14][15][16][17] Our quantitative results demonstrated a reported increase in PCPs' confidence in selecting optimal testing for LTBI (eg, per CDC recommendations, IGRA in individuals with prior BCG vaccination). 25 Participants also reported increased confidence offering shorter-course treatment regimens, such as 4 months of rifampin, which are demonstrated to have a higher rate of completion versus longer regimens. 26 Qualitative results support quantitative metrics and suggested that PCPs made changes to their practice in the realms of LTBI test choice and in offering shorter treatment regimens.
A systematic review suggested that most ECHO program evaluations describe effectiveness for improving lowlevel educational objectives such as recruiting participants and intermediate-level objectives such as improving participants' confidence, but experience difficulty in evaluating the most desirable and highest-level educational objectives such as clinical performance, patient health, and community health outcomes. 11,17 We were able to identify meaningful participant satisfaction (low-level) and changes in knowledge (intermediate-level), as well as subjective measures of increased participant competence (intermediatelevel). Although practice changes that likely impacted patients' health were reported qualitatively, more robust evaluation such as assessing participants' patient outcomes is needed to assess the LTBI ECHO's impact on patient/ population outcomes.
Despite increased confidence, our qualitative interviews revealed logistical challenges. Given differences in PCP and content expert schedules, finding a time that works for both groups is a challenge. In our experience, prioritizing participants' needs is ideal, but may mean that securing content experts will warrant advanced planning. Additionally, we chose to offer a close-ended 6-session course rather than a longitudinal approach as to not overburden PCPs timecommitments. With this approach, we were unable to focus on all populations and topics evenly. However, when the HUB team received feedback regarding these asymmetries in Cohort 1, we made small tweaks to ensure more equal topic distribution in Cohort 2 (eg, additional pediatric cases). Currently, our team has started an advanced longitudinal ECHO in Massachusetts intended to provide opportunities for more in-depth discussion of topics of interest.
This program evaluation has several limitations. First, our cohort sample sizes are small thereby limiting generalizability and interpretation of significance. The low survey response rate may have also introduced a non-response bias. Nurses are critical components of the primary care team and composed 31% of program registrants, but are not represented in the quantitative survey outcomes due to a low number of responses. Future studies should make every effort to include all cadres of ECHO participants. Second, evidence is lacking regarding the impact that the LTBI ECHO may have had on clinical outcomes. Although we asked about self-reported LTBI testing and treatment outcomes, we were unable to account for clinic variability (eg, overall patient visit numbers) and adequately evaluate these data. Therefore, it is unknown if testing and treatment changed before and after the ECHO course.
As states consider their TB elimination plans, our program evaluation suggests that an LTBI ECHO increased PCPs' confidence in LTBI care management and enhanced clinical-care. Our data are consistent with evaluations of other ECHO programs related to other conditions and further support the ECHO Model as an effective means of providing continuing medical education for disease-specific knowledge in primary care. Future research could address the LTBI ECHO's impact on patient-or systemslevel clinical outcomes.