{'fraction_non_alphanumeric': 0.02729439809296782, 'fraction_numerical': 0.013627334127930076, 'mean_word_length': 5.0580024067388685, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The use of patient-reported outcomes becomes standard practice in the routine clinical care of lung–heart transplant patients

Objective: To assess the use of patient-reported outcome (PROs) measures in the routine clinical care of lung–heart transplant patients. We assessed whether the addition of PROs in routine clinical care affected the duration of the consultation and patient’s and clinician’s views. Method: Consecutive lung–heart transplant patients visiting the outpatient clinic, University of Alberta Hospital, completed the Chronic Respiratory Questionnaire (CRQ) and the Health Utilities Index (HUI) on touchscreen computers. Information on the patient’s responses was made available to the members of the transplant team prior to the encounter with the patient. The duration of clinical encounters was noted. At the end of every visit, clinicians completed a questionnaire on the usefulness of having PRO information available. After 6 months patients completed a survey of their experiences. Results: The final patient sample consisted of 172 patients with a mean (SD) age of 52 (13.3) years old; 47% were female; 68% were organ recipients and 32% candidates. The transplant team, comprising four pulmunologists, two nurses, and one pharmacist had an average of 9 years of practical experience in pulmunology. The mean duration of patient–clinician encounters in minutes was 15.15 (4.52). Ninety-eight percent of patients indicated that they would be happy to complete the CRQ and HUI at every clinic visit. Ninety-one percent of the assessments completed by clinicians showed complete satisfaction with the use of PROs in routine practice. Further, the clinicians developed guidelines for the use of PRO information in clinical practice. Conclusions: The incorporation of PRO measures in the routine clinical care of lung–heart transplant patients resulted in a reduction of the duration of patient–clinician encounters. The experience was well accepted by patients and clinicians. We conclude that the routine use of PROs in lung–heart transplant patients has become standard practice.

PROs improves patient-clinician communication and patient management. 24 In the present study the challenges of using PROs are assessed. We described how the routine use of PROs became standard practice in the routine clinical care of lung-heart transplant patients. We assessed patients' and clinicians' views about the implementation of the Chronic Respiratory Questionnaire (CRQ) and Health Utilities Index (HUI) and measured time of the visit. We hypothesized that patients and clinicians would be satisfied with the use of the PROs in routine clinical care and there would be a reduction in the duration of clinical encounters compared to historic norms.
The clinical setting is one for which the routine use of PROs has the potential to improve the quality of care and affect patient management. Both pre-and post-transplant patients present complex management issues reflecting the diverse symptoms associated with their underlying lung disease, its treatment, and comorbidities. In this context, diseasespecific CRQ [25][26][27] and generic preference-based measures such as HUI [28][29][30][31] complement each other by measuring a broad range of health dimensions, including breathing difficulties, pain, ambulation, cognition, and emotional issues that are expected to be relevant.

Study setting and patient population
This prospective study was conducted at the out-patient lung transplant clinic, in a tertiary institution. This service provides clinical care and follow up to patients from four Canadian provinces. The outpatient lung transplant team consisted of four physicians, two nurses, and one pharmacist. The patient sample included both pre-and post-lung transplant people. Patients were excluded if they were aged under 18 years or unable to complete the questionnaires in English. Written information about the study was provided to patients before obtaining informed consent. The follow-up period was 6 months. Ethical approval was obtained from the Health Research Ethics Panel B, file # B-050108, University of Alberta.

Process and data collection
Patients were asked to complete the CRQ and HUI upon arrival at the outpatient clinic. The questionnaires were completed on a touchscreen computer. In a busy clinic, the collection of data using electronic devices (touchscreen, handheld devices, or desk computers) alleviates the burden of using paper-and-pencil questionnaires. The advantages of the use of touchscreen computers over paper-and-pencil questionnaires are: 1) they allow the collection of electronic data that can be stored and scored automatically; 2) they facilitate the availability of the results for presentation to clinicians at each patient's visit in 'real time'.
For this study, Agudo Systems developed a program that allowed for the electronic completion of the CRQ and HUI. The program was specifically developed for this study in consultation with transplant specialists. An additional advantage of this program was that patients' results were available to clinicians immediately in a graphical form prior to the encounter with the patient. The information was printed and added to the patients' medical files together with clinical results from pulmonary function test, a hematological test, and a biochemical test (see Figure 1). Clinicians were trained in the interpretation of the results (further details provided later). At the end of the six months, patients completed an evaluation survey of their views about completing the questionnaires (see Appendix 1). At the end of every visit, clinicians also completed an evaluation survey (see Appendix 2). Clinicians assessed the impact of using the PROs measures in the routine clinical care of the patients by completing a brief 12-item questionnaire. The duration of the clinical encounter was measured. In every consultation room a clock was placed on the desk and activated at the commencement of the patient-clinician encounter and stopped at the end of the encounter. Clinicians filled the time-log form with the time in minutes and number of clinicians present during the encounters (see Appendix 3).
health-related quality of life measures chronic Respiratory Questionnaire (cRQ) The CRQ 25-27 is a disease-specific measure widely used in investigating chronic lung disease. The CRQ contains 20 questions answered on 7-point Likert-type scales. The questions cover four domains: dyspnea, fatigue, emotional function, and mastery. Scores for each of the four domains and for the summary score range from 1 (maximum impairment) to 7 (no impairment), with higher scores signifying lesser severity. A small but clinically important difference in dyspnea, fatigue, emotion, and mastery is 0.5 on a 7-point scale. 25,26 CRQ validity, reliability, and responsiveness have been demonstrated in past studies. 25,27 We used the standardized version of CRQ. 27 health Utilities index (hUi) The 15-item HUI self-assessment self-complete one-week recall questionnaire includes the HUI2 28 and HUI3. 29 HUIs are multi-attribute preference-based measures that are easy to administer, score, and interpret. The HUI2 28 includes seven health attributes (sensation [vision, hearing, and speech], mobility, emotion, cognition, self-care, pain, and fertility) with three to five levels within each attribute for a total of 24,000 unique health states. In this study fertility was not assessed and was assumed to be normal. For both the HUI2 28 and HUI3, 29 single-attribute utility scores (SAUS) are on a scale in which the score for most highly impaired level is 0.00 and the score for normal is 1.00. HUI2 overall scores are on a scale in which the score for the all-worst HUI2 health state is -0.03, dead has a score of 0.00, and perfect health is 1.00. The HUI3 29,30 includes eight attributes (vision, hearing, speech, ambulation, dexterity, cognition, emotion, pain, and discomfort) with five or six levels for each attribute. HUI3 overall scores are on a scale in which the all-worst HUI3 state has a score of -0.36, dead is 0.00, and perfect health is 1.00. 30 Changes of 0.03 or more in overall HUI scores and 0.05 or more in single-attribute scores are clearly clinically important. 31 The HUIs have been validated in several past studies. 30 The HUIs were selected for several reasons. First, during initial discussions with clinicians, the clinicians suggested the use of a generic measure, arguing that they routinely discuss the content of specific measures in their encounters with patients. Second, in the context of transplantation, it was decided that it would be useful to adopt a preference-based measure that could also be used to estimate quality-adjusted survival, a fundamental outcome of transplant programs. Furthermore, the HUIs are useful measures because they both describe a great number of health states, and capture the severity of the disease and burden of side-effects associated with drugs and other treatments. For instance, post-transplant patients receiving immunosuppressant drugs may suffer vision and speech problems, depression, and cognitive deficits. Also, pre-transplant patients with symptoms such as fatigue and breathing limitations will limit ambulation and self-care.

Patient chart
The chart compiles the results from the CRQ and HUI (see Figure 1). The left side of the card displays the graphical representation of the measures with additional   Information about clinically important differences (change measurement scores detected by patients as important) is added. The right side of the chart displays comments from clinicians. The CRQ score card displays results for each of the four domains to assist in the interpretation of the scores. Because information about previous results was easily retrievable from the electronic files, clinicians were able to compare current and previous results and track a patient's progress. The HUI score card includes a display of single-attribute scores for HUI2 emotion, HUI2 self-care, and each of the eight HUI3 attributes. The two attributes from HUI2 were selected because of their high degree of relevance in the context of lung transplant. To assist in the interpretation of overall HUI utility scores, ranges of overall scores are classified in the following disability categories: no disability; mild disability; moderate disability; and severe disability. Similarly, each level in each attribute within both of the HUI systems corresponds to one of the categories: no, mild, moderate, or severe disability. Single-attribute and overall scores were displayed using a color coded system in which green signifies normal (no problem), yellow signifies a mild problem, blue a moderate problem, and red a severe problem.

Training in the interpretation of the patient chart
Training in the interpretation of CRQ and HUI results was provided to clinicians on a regular basis. Training was provided by one of the authors (MS). At the beginning of the study, formal educational sessions were conducted for the members of the team and one-on-one sessions were held with physicians. Clinicians completed the questionnaires and a chart or score card was printed to facilitate and guide the interpretation of the scores. Patient case studies were presented linking the clinical data to the CRQ and HUI results. Clinicians used the patient PRO information to identify patient's problems and to monitor patient's progression. Clinicians developed guidelines for the use of the patient chart in the outpatient clinic.

Descriptive measures Patients' sociodemographic and clinical characteristics
At the first study visit (baseline assessment) the patients completed a brief sociodemographic questionnaire. The purpose was to provide a description of sociodemographic characteristics in this patient population. Items included age, gender, level of education; working status, and a list of comorbidities (see Appendix 4).
A chart review has been designed to collect clinical data: type of transplant, date since transplant, current drug therapy, pulmonary function tests results, and six minute walk test (see Appendix 5).

Patient evaluation questionnaire
Patients assessed the ease, usefulness, and satisfaction of completing the PROs at every outpatient clinic visit. The patient-evaluation questionnaire included 10 questions. Responses are on a 5-point Likert-type scale: definitely not, probably not, not sure, probably yes, definitely yes (see Appendix 1). Patients completed this questionnaire at the end of 6 months of follow-up.

clinician evaluation questionnaire
Clinicians assessed the usefulness of the use of PROs in their routine clinical care by completing a 12-item questionnaire. Responses are rated on a 5-point Likert scale: never, seldom, sometimes, often, and always. This questionnaire assesses a variety of potential effects including adding new information on patient's health status, helping with history taking, adding clinical relevance, and helping in the clinical management of patients. This questionnaire also contains a global satisfaction question (see Appendix 2).

Duration of clinical encounters
The duration of clinical encounters was measured in the previous trial conducted in the lung transplant clinic at the University of Alberta. The average time of consultation was 17.30 minutes. We expected that the use of PROs would enhance patient-clinician communication by conveying important information in a more efficient manner than in usual care. We therefore expected a reduction in the duration of clinical encounters. The duration of consultations was measured at every visit and entered in the time log form (see Appendix 3) at the end of the visit. Also, the number of clinicians per encounter was entered in this form (some patients were only seen by pulmonologists and others by pharmacist and pulmonologist or by the nurse and pulmonologist). This form was kept in the patients chart.

Data analysis
To assess challenges to the implementation of the PROs in routine clinical care the following analyses were conducted. Descriptive information on the CRQ and HUI is reported. Descriptive statistics were used to describe the

97
The use of PROs becomes standard practice characteristics of the patients and clinicians. Data collected from the patients' and clinicians evaluation surveys was used to describe patient's and clinician's usefulness with the implementation of the PROs. The mean global clinician evaluation score was compared among the three groups of clinicians (pulmonologists, pharmacist, and nurses), using analysis of variance (ANOVA). The mean time (minutes) of the clinical encounter was compared to the mean time of the clinical encounter from the previous study. To explore the possibility that the encounter involving two clinicians and a patient will take longer than the encounter involving one clinician and a patient, the mean time for the two types of clinical encounters was compared. The statistical analyses were conducted using SPSS (V. 15.00; SPSS Inc, Chicago, IL).

characteristics of patients
One hundred ninety patients were invited to participate in the study; 186 agreed (response rate 97.9%). Fourteen patients were lost (4 died and 10 post-transplant patients didn't schedule visits during the 6-month follow-up); analyses are based on data for 172 patients (90.5% response rate). Table 1

characteristics of clinicians
All seven members of the lung transplant out-patient clinic team participated in the study. Two of the seven of the transplant team were aged over 51 years with a mean of years of training of 9 years and had practiced for an average of 12 years. There were two female nurses with an average of 10 years of experience and four pulmunologists (range of years of experience in lung transplant 2 to 12). Table 2 displays the mean (standard deviation) of the CRQ scores for the entire cohort of patients. Fatigue was the most affected domain (4.30 ± 1.50), followed by dyspnea, emotion, and mastery. The mean HUI3 overall score of 0.70 for the patients indicates moderate to severe disability. Overall scores ranged from -0.21 to 1.00. HUI3 ambulation and HUI3 pain (0.81 and 0.87, respectively) were the most severely affected attributes.

Patient reported outcome results
Patients' views were positive (see Table 3). All patients enrolled were happy to complete the questionnaires; only 3% of the patients (two pre-transplant and three long-term post-transplant patients) were not willing to complete the questionnaires as part of their routine visits to the clinics. Most of the patients thought that the completion of the questionnaires didn't make their visit difficult and that it wasn't time consuming. Ninety-one percent of the patients   confirmed that the questionnaires asked the right questions but 62% thought that some of the questions were irrelevant to their condition. One patient found the questions regarding mental health upsetting. Patients were aware that the PROs information was kept in their medical records and it could be accessed at any time; 59% of the patients kept the print out of the results as a way to follow their progression. Clinicians were satisfied with the evaluation in 91% of cases (see Table 4). The information provided confirmed the diagnosis in 59% of the cases and added new information in over 92% of the visits. In more than 90% of the visits the information provided contributed to overall assessment, helped patient-clinician communication, and helped in the clinical management of the patient. Clinicians thought that in 26% of the visits the use of PROs prolonged the duration. The mean overall clinician satisfaction was 6.23 ± 0.62. Mean overall clinician satisfaction was statistically significant (analysis of covariance; ANCOVA, P , 0.001) for the differences among pulmunologists, pharmacist and nurses (6.30 ± 0.62, 6.20 ± 0.63, 6.00 ± 0.60, respectively).
The mean duration of visits in minutes was 15.30 ± 4.12 minutes. There was a statistically significant difference (P = 0.001) between the historical cohort (17.30 ± 12.14) and the present cohort. Of the 427 visits, 303 visits occurred between one pulmonologist and a patient with a mean time of 15.14 ± 4.37 minutes. Other types of encounters with a patient included nurse-pulmonologist or pharmacistpulmonologist and accounted for 167 of the visits. The mean time of the visit was 15.57 ± 4.11 minutes. The difference in the duration of the two categories of visits was not statistically significant (P = 0.34).

Discussion
This study is the first in which the challenges using CRQ and HUI in routine clinical lung transplant patients were assessed. We were able to describe patients' and clinicians' views of the assessments as positive, and to detect a reduction in duration of visits. Our results are important because they demonstrate that it is possible to overcome the  barriers identified that impede the use of PROs in routine clinical care. Deyo and Patrick 1 summarized previous experience of the use of PROs measures in clinical trials. For the healthcare providers the main concerns were related to the clinical effectiveness of using PROs measures, lack of familiarity with the measures and the interpretation of their scores, and the difficulty of fitting the administration and scoring of the measures into the office ecosystem (lack of time and resources). The main concerns of patients were related to the disclosure of information (insurance status) and understanding and interpreting the questionnaires.
In trying to overcome the barriers, clinicians were consulted about the selection of the PROs. The involvement of the clinicians helped with the skepticism about the validity of PRO data. Clinicians were familiar with the CRQ because it is a widely used specific lung disease measure. [25][26][27] They were also familiar with the HUI because of their prior experience using the measure. 23,24 We also trained clinicians to interpret and understand the magnitude of treatment effect. The training was based on a previous study 24 where clinicians completed the questionnaires to became familiar with the content and learn about the clinically important difference (the smallest change in scores detected by the patient as important) that helped with the interpretation of the scores.
The training for the present study was improved by adding frequent case-study presentations to the team. We presented patients' PROs results to explain change as result of disease progression or modifications in medical treatment. We found that presenting longitudinal data helped clinicians to understand the clinical effectiveness of using PROs.
The difficulty of fitting the administration and scoring of the measures into the office ecosystem was alleviated by the use of electronic data collection. We used touch-screen computers connected to printers in the outpatient clinics. Rose and Bezjak 12 presented an overview of the logistics of collecting PROs in clinical practice, including comparison among different types of computerized technologies. The authors suggested that one of the disadvantages is the lack of familiarity with computers. Despite the fact that most of the patients only had a high school education and that many came from rural areas, they did not have problems completing the questionnaires electronically. Patients suggested that the completion of the questionnaires did not make their visit longer and that they were happy to complete the questionnaires at every clinic visit. Patients were highly satisfied with the consultations and felt that the questionnaires were a useful tool to tell the doctors about their problems. The clinicians perceived that the quality of life data broadened the range of the clinical inquiry and helped them identify issues for discussion. Clinicians have found PROs useful and not disruptive to their practices.
The results are in keeping with previous studies. [7][8][9][10]13,15 Most of the patients were willing to participate in the study and agreed that the completion of the questionnaires did not make their visit more difficult. Furthermore, clinicians' evaluations of the assessments exceeded their expectations. At the end of the study, clinicians' evaluations revealed the usefulness of the PROs in the routine clinical care of lung transplant patients. Clinicians agreed that the patient chart confirmed diagnoses, facilitated history taking, communication, added new information, and helped with the management of patients. Detmar et al 15 learned that the PROs data stimulated physicians to initiate discussions on wider aspects of functioning. Similarly, Taenzer et al 7 and Velikova 10 showed that the computer measurement was well accepted by patients.
We assessed the barrier related to time constraint and detected a reduction in the duration of visits. It needs to be highlighted that the inclusion of PROs in the clinic did not make the visit longer. Clinicians commented that the information provided allowed them to focus on problems highlighted by the questionnaire responses, conveying more information in a timely manner. Time constraint has not been explored in past studies.
The cost of the implementation was covered by two grants (University of Alberta Hospital Transplantation Value Fund and Roche) that covered the materials and personnel expenses. Reimbursements to clinicians for their time spent in training, attending meetings, and using the PROs was not regarded as a problem in this particular setting organized in an academic payment plan. Perhaps the situation may differ in other settings. It is important to note that the expenses are a small fraction of the local health budget compared to other technologies used in the clinical care of patients.
The present study had several limitations. First, the study was conducted in a highly specialized clinic and therefore results may not be generalizable to other clinical settings. Secondly, patients' evaluations may be influenced by their gratitude towards the transplant team either for being accepted onto the waiting transplant list or, for those who underwent transplantation, having been given the new organ. Furthermore, the patients were closely followed at a highly specialized outpatient clinic by a caring team of clinicians. Also, the clinician team was willing to participate in the study despite the potential barriers in the use of PROs in routine clinical care. The sample size of clinicians was not very large and it may be interesting for future researchers to include a larger number of clinicians in their studies. Future researchers should conduct studies in different settings with larger number of clinicians from different specialties, in order to evaluate further the use of PROs in clinical care.

Conclusions
The routine clinical use of PROs measures in the clinic was well accepted by patients and clinicians and was associated with a reduction of the duration of visits. Overall, the use of PROs in the lung transplant out-patient clinics has become standard practice.