{'fraction_non_alphanumeric': 0.028018942383583267, 'fraction_numerical': 0.01254932912391476, 'mean_word_length': 5.340005003752815, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Feasibility and Acceptability of Implementing Site-Specific Patient-Reported Outcome Measure in Head and Neck Cancer Clinics: A Prospective Institutional Study

Purpose To assess the feasibility and acceptability of implementing the MD Anderson Symptom Inventory–Head and Neck (MDASI-HN) module in cancer clinics and its effect on patient-reported experience. Methods We conducted a prospective, longitudinal study at a tertiary cancer institution between September 2020 and August 2021. Patients with newly diagnosed head and neck (HN) cancer who were evaluated to receive radiation therapy with or without chemotherapy and could communicate in English were approached to participate. The primary outcome was feasibility and acceptability of the MDASI-HN implementation in the radiation oncology department assessed by patient and provider exit surveys. Secondary outcomes were patient-reported experience as recorded by a shortened Your Voice Matters survey (YVM) in 2 cohorts pre- and post-MDASI-HN implementation and symptom scores. Descriptive statistics were used for exit surveys and symptom scores. Mann-Whitney tests were used to assess differences in positive responses between pre- and postimplementation for each YVM question. Cochrane-Armitage tests were used to examine changes in patient-reported experience over time. Results Fifty-one patients were enrolled in the postimplementation cohort and 29 (60%) responded to the exit survey. Eighty-nine percent of patients reported that MDASI-HN made it easier to remember symptoms, and 86% recommend its use in routine care. Four of the 5 radiation oncology HN providers (80%) responded to exit surveys; 75% felt the MDASI-HN provided clinically relevant information, improved communication with patients, and did not increase clinic time. The overall patient-reported experience was not affected by the implementation (P = .82). The probability of positive responses over time was significant (P = .025) in the clinic coordination domain for the postimplementation cohort. Conclusions Implementation of MDASI-HN was feasible and acceptable by patients and providers. Although the overall patient-reported experience was not affected by implementation, some aspects improved as treatment progressed.


Introduction
Person-centered, high-quality care entails placing the patient at the center of health care delivery, and it is a priority in many national health care policies. 1-4 It is a multidimensional process that transcends cancer control or eradication to evoke a personalized reduction of the effect of cancer and its treatment on patients and addressing their needs throughout the health care delivery process. 5 In head and neck (HN) cancers where most patients are treated with aggressive therapy, toxicity builds up, leading to significant physical and/or emotional symptoms and a detrimental effect on the quality of life. 6,7 With better cancer control outcomes in the HN, there is a growing interest in evaluating the treatment effect and quality of life. 8,9 One method of evaluation is through patient-reported outcome measures (PROMs), which aim to capture the effect of disease, its associated treatment, and any other interventions on health status from the patient's perspective. 10 There are several advantages to incorporating PROMs in outpatient oncology clinics including potential improvement in survival, 11,12 reduction in health resources utilization, 13,14 and symptom monitoring through enhanced patient−provider communication. 15 In HN patients treated with chemoradiotherapy, it has been observed that provider-reported side effects are less severe than patient selfreports, 16 which further highlights the importance of systematically incorporating the patient's perspective into the health encounter. The MD Anderson Symptom Inventory −Head and Neck Module (MDASI-HN) is a validated sitespecific PROM 17 that captures symptoms related to HN cancer, which are not typically included in the generic PROMs that capture symptoms common to most cancers. Although the MDASI-HN has been used in clinical trials 18 and validated in several languages, 19 implementation studies in routine clinical practice are lacking. 20 Additionally, there is no information pertaining to the effect of MDASI-HN implementation on patient-reported experience while receiving radiation therapy.
Patient-reported experience measures (PREMs) are surveys that capture the views of patients on whether their needs have been addressed throughout the health care delivery process. 21 PREMs can be used to assess and benchmark the health care system performance, 22 yet its uptake is slow. 23 Your Voice Matters Survey (YVM) is a validated PREM 24 that can measure patient-reported experience in real time. 25 Given that the effect of using the MDASI-HN in clinical care on patient-reported experience is unknown, we hypothesized that successful implementation of MDASI-HN would result in an improvement of patient-reported experience through enhanced patient−provider communication; therefore, the aim of this study is to assess the feasibility and acceptability of implementing the MDASI-HN in routine care and its effect on PREMs during radiation therapy.

Methods and Materials Experimental design
This was a prospective, institutional pilot study evaluating the integration of the MDASI-HN into longitudinal outpatient clinics to assess acute care within the radiation oncology department. The study ran in 2 phases. The first phase captured patient-reported experience before introducing MDASI-HN (the preimplementation cohort). In the second phase, the MDASI-HN was implemented and patients were asked to complete the MDASI-HN and then the patient-reported experience (the postimplementation cohort). Time points of data collection included the initial radiation oncology consultation, weekly treatmentreview appointment, and follow-up appointments with the nurse practitioner at end of radiation therapy and with the radiation oncologist 6 to 16 weeks posttreatment (Fig. E1). The ethics review board approved this study (protocol identifying number: HREBA.CC-18−0588). This study is reported according to the Standards for Reporting Implementation Studies statement. 26

Study context
This study was conducted at a tertiary cancer center with a multidisciplinary team of care providers. Patients attending the HN clinic are seen by radiation oncologists, surgeons, and nurses, as well as dieticians, speech−language pathologists, and psychosocial support, if required. Before this study, a nurse practitioner clinic was established to provide symptom support to HN patients undergoing concurrent chemoradiation therapy and patients who require additional symptom support. The team responsible for patient care during radiation therapy is composed of 4 radiation oncologists and 1 nurse practitioner. Concurrent chemoradiation patients were typically scheduled to see the nurse practitioner instead of the radiation oncologist at the second, third, fifth, sixth, and seventh treatment review appointments, as well as the first Monday (extra visit) following radiation treatment completion. Patients undergoing radiation therapy alone typically see their radiation oncologist weekly but could be referred to the nurse practitioner if additional support was needed at any time. Each patient's radiation treatment duration varied and could take up to 7 consecutive weeks.
The entirety of the project was completed during the COVID-19 pandemic. Ethics approvals and modifications were obtained to begin the project in September 2020. To accommodate to changing institutional pandemic research regulations, all study-related follow-up post initial recruitment was completed virtually. Recruitment for the preimplementation cohort was conducted between September and December 2020. Shortly after recruitment began, Alberta entered its second wave of COVID-19, peaking in December 2020. Recruitment for the postimplementation cohort began in January 2021 through April 2021, and follow-up concluded in August 2021. During this time, Alberta entered the third wave of the pandemic in March 2021.

Patient recruitment
Patients with a confirmed HN cancer diagnosis were eligible to participate if they were going to receive radiation therapy alone or with chemoradiotherapy. Also, patients must be ≥18 years of age, able to read and speak English, and willing to participate in this study.
A convenience sample was used to recruit patients. The research coordinator prescreened HN clinic lists to identify eligible patients at the initial consultation appointments. Patients were then approached by the research coordinator and/or radiation oncology research fellow. Patients who agreed to participate received information about the study details, provided informed consent, and completed the questionnaires.

Surveys and data collection Patient-reported outcome measures
The Edmonton Symptom Assessment Scale−Revised Edition (ESAS-r) is a generic measure that is a part of the standard of care at our institution. The ESAS-r collects information on 9 general core symptoms. Response options range from 0 (the patient does not have the symptom) to 10 (the symptom is at its worst). 27 The MDASI-HN was selected for implementation because it has good symptom coverage, usability, and psychometric properties. 28 It assesses 13 core symptoms, 9 symptoms relevant to HN treatment, and 6 items regarding symptoms' interference with daily living. Response options to the symptom questions ranged from 0 (no symptom) to 10 (the symptom is as bad as the patient could imagine).
Patients were asked to complete the symptom forms in the waiting room before their consultation appointment on a tablet device. For all subsequent treatment reviews and follow-up appointments, MDASI-HN completion was done virtually 24 hours before the appointment, either via email or over the phone. Data are collected and stored on a secure research database (REDCap). To facilitate clear and efficient interpretation of longitudinal symptom changes by clinicians, an electronic dashboard was created using third-party software (Tableau), which was connected to REDCap (Fig. E2). Clinicians could access Tableau to review all completed MDASI-HN forms and view symptom trends over time for each participant.

Patient-reported experience measures
Our research team, including radiation oncologists and health services researchers, selected 6 out of 28 YVM survey questions that assess physical comfort, emotional support, coordination of care, communication domains, and overall experience. The team felt these items would most likely be affected by MDASI-HN implementation. Other items were excluded (eg, questions about the check-in process at reception) to shorten the survey and minimize survey fatigue. An additional question was added for the postimplementation cohort asking patients: "To what extent did your health care provider review your responses to the symptom form you completed before your appointment?" Response options to YVM questions included excellent, very good, good, fair, poor, and not applicable. A qualitative (open-text) question was included to collect additional feedback from patients. YVM forms at consultations included additional demographic questions, and YVM forms during treatment included a question regarding the treatment(s) the patient is receiving. Cancer stage and treatment intent was collected through electronic medical records.
Patients completed a paper copy of the YVM at the end of their consultation appointment at the clinic in both preand postimplementation cohorts. All other follow-ups of the YVM data collection were completed virtually either via email or over the phone with the research coordinator. Clinicians were blinded to patient experience data.

Patient and provider exit surveys
Exit surveys were administered to obtain patient and provider feedback on using the MDASI-HN. The exit surveys have been used previously in other studies. 29 The patient and provider exit surveys included 10 and 13 questions respectively, with response options ranging from 1 (not at all) to 4 (very much). Two open-text questions were also included for both exit surveys. Exit surveys were completed at the first posttreatment follow-up (patients) and at end of study (providers) either virtually or via email or telephone.

Outcome definition and analysis
Descriptive statistics, including frequencies and proportions, were calculated. The results of the exit surveys Advances in Radiation Oncology: November−December 2022 Feasibility of implementing site-specific PROM were used to evaluate the feasibility and acceptability of implementation. Acceptability was defined as the proportion of participants recommending MDASI-HN in routine care and its helpfulness in discussing symptoms. 30 Feasibility was defined as the success of the integration of Tableau software for symptoms' display. 30 The proportion of positive responses used to indicate the successful implementation was chosen to be >60%. Response options of 1 (not at all) and 2 were considered negative, and responses of 3 and 4 (very much) were considered positive. Open-text comments were coded and grouped into themes using content analysis. Changes in average symptom scores were examined over the treatment trajectory for patients who completed the MDASI-HN at each time point. Patient-reported experience was compared between the pre-and postimplementation cohorts. For the YVM quantitative questions, percent positive scores (the proportion of positive experience responses) were calculated using the equation: Percent positive score (%) = Positive/(Positive + Negative) £ 100. The percent positive score for each question and time point were plotted to examine the trajectory of patient experience throughout radiation therapy between both cohorts. Mann-Whitney tests were calculated to test for differences in percent positive scores between pre-and postimplementation for each YVM question. Cochran-Armitage trend tests investigated trends in positive responses over our study time points for both pre-and postimplementation YVM data. We used SAS Enterprise version 7.1 (SAS Institute, Inc).

Results
A total of 32 patients in the preimplementation and 51 patients in the postimplementation cohorts were enrolled. Patients' characteristics are summarized in Table 1. The proportion of patients who completed the questionnaires had decreased as treatment progressed for both YVM and MDASI-HN (Fig. 1). In the preimplementation cohort, YVM completion ranged from 78.1% to 33.3% from the first to seventh treatment review, respectively, and from 68.6% to 38.1% in similar time points for the postimplementation cohort. Over 80% of items were answered for all returned YVM forms during treatment reviews and follow-up appointments for both cohorts. For MDASI-HN, 442 surveys were administered to participants with 279 surveys completed (63.1%) with a range from 72.5% to 28.6% from the first to the seventh review, respectively. Over 90% of participants with returned forms completed at least 80% of survey items at each time point.

Exit surveys
A total of 29 (60.4%) participants completed exit surveys. Three patients deceased before follow-up appointments and were excluded from the denominator of the completion rate. Most participants found the MDASI-HN was easy to use (86.2%) as did all participants who completed the questionnaires via email (100%). Most patients would be willing to complete the symptom questionnaire at every clinic visit (72.4%) and would recommend the symptom questionnaire be part of routine care (86.2%), indicating acceptability. Most participants indicated that the questionnaire made it easier to remember and report symptoms at clinic visits (89.3%) (Fig. 2).
Most participants (96.5%) completing the exit survey provided open-text feedback. Frequent comments regarded positive feedback and a willingness to complete the symptom questionnaire (42.9%) and commending the radiation care team (21.4%). Other feedback included inconsistency/not enough discussion of the MDASI-HN content during reviews (21.4%), the MDASI-HN was too long (17.9%), and the form was difficult/not completed when patients were feeling very ill (14.3%).
Four of 5 (80%) of health care providers completed the provider exit survey. Most providers (75%) reported that Tableau was easy to use and navigate to review MDASI-HN results. Most providers (75%) thought the MDASI-HN information was relevant, improved communication, and the included HN-specific symptoms were more helpful than collecting generic symptoms alone. All providers reported that the data collected was accurate (Fig. 3).
Open-text comments expressed that most symptoms were already assessed in the clinic before the implementation with no significant addition to usual care, a preference to review paper forms in the clinic, and a suggestion that integration of a symptom display into an electronic medical system would improve the use of the dashboard.

Patient-reported experience
We found no difference in the overall patient-reported experience (P = .820) when comparing the pre-and postimplementation cohorts. Favorable patient-reported experiences included discussing physical symptoms, health care providers spending enough time with patients, and quality of care (Fig. 4). After the MDASI-HN implementation, percent positive scores for discussing physical symptoms during appointments were higher than the preimplementation cohort from consult (93% vs 83%) to treatment review 4 (100% vs 94%), although not statically significant (P = .186). Most patients reported that health care providers reviewed their responses to completed symptom forms during their appointment (72.3%).
In the postimplementation cohort, the probability of positive responses for discussing physical symptoms increased over time (P = .003), as discussing emotional worries (P = .007), time spent during appointments (P = .008), and the quality of care (P = .011) for the preimplementation cohort. On the other hand, the probability of positive responses increased with time for discussing physical symptoms (P = .031), discussing emotional worries (P = .028), and the clinic team working together/being coordinated (P = .025).

Symptom trajectories
The symptoms with the highest (worst) average scores included problems tasting food, dry mouth, mucous in the mouth/throat, and difficulty swallowing (all from the MDASI-HN), as well as a lack of appetite and tiredness (both from the ESAS-r). These symptoms worsened as treatment progressed and peaked at the end of treatment.

Discussion
Implementation of MDASI-HN in HN cancer clinics was feasible and acceptable to patients and providers. Although some aspects of patient-reported experience improved as treatment progressed despite worsening symptoms, the overall experience did not improve with MDASI-HN implementation.
Implementation studies have typically evaluated their success through exit surveys that captured patients' and providers' views about the implementation process. 29,31 Although our study followed a similar approach, it is unique in incorporating a real-time PREM to evaluate whether patients' needs were addressed. Our study is important as it shows that the implementation of the MDASI-HN may be associated with an improved experience for certain aspects but not the overall experience. One reason for not improving the overall experience may be due to the ceiling effect. 32 The MDASI-HN showed an upward trend in the average symptom scores for all items that peaked around the end of treatment. Most patients reported that the MDASI-HN made it easy to remember and report symptoms, which is important whenever cancer-or treatment-related symptom burden is significant. Our study provides novel findings examining the trends in experience during radiation therapy, with or without concurrent chemotherapy. We found that the probability of a positive experience increases over time for discussing physical symptoms and emotional worries for Advances in Radiation Oncology: November−December 2022 Feasibility of implementing site-specific PROM both pre-and postimplementation cohorts. These findings are interesting as most HN patients will encounter worsening treatment side effects as treatment progresses and their needs will grow, which might be unmet, potentially leading to a worse experience. However, we found that many aspects of patient experience improved as treatment progressed. These results likely reflect the strong, relational aspects of care that develop throughout treatment between the patient and their providers. Comments from the exit surveys support this insight, as many patients conveyed their gratitude for their radiation therapy team. We additionally found that the probability of a positive experience with clinic coordination/teamwork improved with time for the postimplementation cohort, which may indicate some helpfulness of the MDASI-HN with coordinating complex patient care as treatment progresses. For both pre-and postimplementation cohorts, we did not find significant trends for overall patient experience improving with time. This likely reflects the complexity and multitude of factors that contribute to the overall patient experience.  Programs aiming to implement PROMs into their programs need to consider facilitators and barriers related to their institution. 33 Our study had many facilitators, including clinic champions and a research support program that used a third-party software to facilitate easier display and longitudinal tracking of symptoms. Further, clinicians were given a presentation on how to use the software. Clinic workflow was examined at the planning phase and to mitigate the competing demands on time that affect the clinic workflow, patients were asked to complete the MDASI-HN after they check in and before being seen in the clinic. Despite careful planning, our study identified several barriers. One barrier is the need for a separate login for the electronic third-party software. This created time and effort demands for some providers. To overcome this known barrier, the department is taking steps to have an electronic software to display symptoms that is integrated into the electronic health record system.
An additional barrier included a lack of consistency in the collection of patient-reported experiences, including difficulty reaching patients virtually. We also identified an inconsistent approach to discussing completed MDASI-HN during appointments. Some open-text comments from the patient exit survey reported little discussion from the provider regarding their symptom form and would have preferred more. This finding emphasizes the importance of acknowledging a patient's effort to complete a PROM and explicitly referencing it during the encounter to ensure strong implementation. Further, it highlights the need for future studies to explore the reasons behind the inconsistent PROM discussion and may represent an opportunity to improve the process of care.
The implications of our study include advancing highquality, person-centered care in accordance with national strategies. 1-4 Also, successful implementation can help with objective symptom tracking when multiple providers   Feasibility of implementing site-specific PROM are involved, tailoring treatments, 34 future research comparing real-world PROM data to what was reported in clinical trials, 20 and outcome predictions. 35,36 The limitation of this study includes the potential effect of the COVID-19 pandemic on MDASI-HN symptom reporting, although one study showed symptom reporting with MDASI-HN did not change in the pandemic in comparison with the prepandemic era. 37 Also, there was an effect on the YVM completion rates, and although we were able to modify our data collection to be completed virtually via phone or email due to COVID-19 restrictions, we did experience a relatively high volume of missing data, especially toward the latter part of treatment that may have been mitigated if data collection was completed in person. Further, having a larger sample size, especially toward the end of treatment, would have provided greater accuracy in our analysis of experience and symptom data. Therefore, our study might not fully represent the PREMs toward the end of treatment given the low completion rate. The need to convert to data collection by phone or email also limits our ability to generalize our results to a nonpandemic environment where all data would be collected in the waiting room. Finally, this study represents the experience of implementation within a single institution with a HN radiation team of 5 providers, which might limit the generalizability of the results. However, our study provides important lessons for departments looking into the implementation of MDASI-HN into routine clinical care. For example, the integration of the PROM within the electronic medical records system.

Conclusion
Implementation of the MDASI-HN was feasible and acceptable in HN clinics. Although the implementation did not change the overall patient-reported experience, trends for improving experience were observed as treatment progressed despite worsening of symptoms.