{'fraction_non_alphanumeric': 0.030977376903740944, 'fraction_numerical': 0.008132485583321011, 'mean_word_length': 5.159608378870674, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Consensus Set of Outcomes for Parkinson’s Disease from the International Consortium for Health Outcomes Measurement

Background Parkinson’s disease (PD) is a progressive neurodegenerative condition that is expected to double in prevalence due to demographic shifts. Value-based healthcare is a proposed strategy to improve outcomes and decrease costs. To move towards an actual value-based health care system, condition-specific outcomes that are meaningful to patients are essential. Objective Propose a global consensus standard set of outcome measures for PD. Methods Established methods for outcome measure development were applied, as outlined and used previously by the International Consortium for Health Outcomes Measurement (ICHOM). An international group, representing both patients and experts from the fields of neurology, psychiatry, nursing, and existing outcome measurement efforts, was convened. The group participated in six teleconferences over a six-month period, reviewed existing data and practices, and ultimately proposed a standard set of measures by which patients should be tracked, and how often data should be collected. Results The standard set applies to all cases of idiopathic PD, and includes assessments of motor and non-motor symptoms, ability to work, PD-related health status, and hospital admissions. Baseline demographic and clinical variables are included to enable case mix adjustment. Conclusions The Standard Set is now ready for use and pilot testing in the clinical setting. Ultimately, we believe that using the set of outcomes proposed here will allow clinicians and scientists across the world to document, report, and compare PD-related outcomes in a standardized fashion. Such international benchmarks will improve our understanding of the disease course and allow for identification of ‘best practices’, ultimately leading to better informed treatment decisions.


INTRODUCTION
Parkinson's disease (PD) is a common and progressive neurodegenerative disease [1]. In the USA, PD has an estimated prevalence of 0.3% and an estimated healthcare cost per patient of 10,000 USD/year [2]. Prevalence and costs are similar in Europe [3]. Due to the aging global population, the prevalence of PD is expected to increase significantly [4], leading to greater disease-associated burden and higher care expenditures. Optimizing the quality of PD care and minimizing the expense of care delivery are therefore essential.
Increasing value, defined as a patient's outcomes divided by the cost to achieve those outcomes, has been proposed as a mechanism to improve the quality of care [5]. A systematic measurement of outcomes can guide improvement and enable dissemination of best practices. In order to move towards an actual value-based health care system, having condition-specific outcomes that are meaningful to patients and their care providers is crucial. Transparency regarding outcomes and costs is essential to help reduce unwanted variations in healthcare delivery, and to increase the overall quality of care. This need has been recognized in the PD community for some time. Efforts to identify outcomes that are meaningful to patients and caregivers have led to the establishment of various national assessment programs [6][7][8][9]. which in turn obscures our ability to learn from best practices, a necessary step to improve global healthcare.
The International Consortium for Health Outcomes Measurement (ICHOM) was formed to develop global consensus sets of outcomes that reflect patients' concerns and experiences. ICHOM has already developed international sets of outcomes for 21 medical conditions [10]. We here report the results of an ICHOM initiative to develop a similar set of outcomes for PD. To achieve this, ICHOM brought together an International Working Group, representing patients, neurology, psychiatry, nursing and existing outcome measurement efforts, to develop a parsimonious standard set of outcome indices for PD, with the aim of proposing the product for international use. This paper describes the development process and the resultant set.

Working group
The formation of the Working Group was based on the principles of previous ICHOM working groups [11]. The PD Working Group consisted of 12 members from eight countries (USA, Canada, UK, Spain, Italy, Germany, Netherlands, and Sweden) and included expert neurologists (n = 9), a psychiatrist, and a nurse specializing in PD, as well as an experienced patient advocate (Table 1). Working Group members were identified by reviewing authors of leading papers on PD care quality, and by identifying members of international patient advocacy groups, leading international PD scientific organizations, and leading physicians in existing national and international quality measurement efforts.

Process
Following the process used in earlier ICHOM work [10,11], a modified Delphi technique was employed to define the outcomes and case-mix variables. Case mix variables are defined as those variables that capture the state of the patient independent of the medical condition for which they are being treated. This includes demographic factors, health status (e.g. co-morbidities) and treatments. The process is a structured, consensus-driven approach, with teleconferences and post-teleconference surveys to reach decisions. Proposals for each teleconference were generated in advance by a core ICHOM project team (RU, TAK, PdR). These were based on a literature review of existing guidelines and standards, as well as individual interviews with each Working Group member.
The Working Group was officially announced in December 2013 and launched with an inperson meeting at the conference of the International Association of Parkinsonism and Related Disorders (IAPRD). This was followed by five 75-minute teleconferences, which took place every month between January and May 2014. All of these teleconferences were followed by a survey of the Working Group members to make decisions on key discussion areas. A 2/3 majority was required, being a commonly used threshold for Delphi and modified Delphi processes, on each survey question to reach consensus. Shifting the threshold a bit did not have an impact on the selection process. When a 2/3 majority was not reached, the topic was brought up for re-discussion at the following teleconference. The standard set of outcomes was then launched at the International Parkinson and Movement Disorder Society (MDS) Conference in June 2014.
The process began with defining the scope of the Working Group by deciding which causes of parkinsonism to include in the set. Subsequently, key outcome domains that are meaningful to patients were identified based on relevant literature and outcome measurement programmes [6][7][8][9][10][11]. These were then reviewed with each Working Group member individually to determine if additional domains, not identified by the search, should be considered. The resultant list of outcome domains was then organized based on four criteria. Each criterion was rated on a Likert scale of 1-4, where one was the lowest and four was the highest score given: (1) Frequency of the outcome domain in the patient population -an important consideration for a set that aims to be parsimonious; (2) Impact of the outcome domain on the patient -an essential consideration for a set that aims to reflect what is most meaningful to patients; (3) Preventability/treatability of the outcome domain -a necessary consideration for a set that aims to be used in the clinic to generate meaningful data on which clinicians can act to modify their practice; and (4) Feasibility to capture the outcome domain in clinical practice -this is essential as the set is designed to be used in routine clinical practice. This formed the basis for the first teleconference discussion.
Once the outcome domains were decided, the tools for data collection were determined. Relevant scales or items were identified and prioritized using specific criteria. Again, each criterion was rated on a Likert scale of 1-4, where one was the lowest and four was the highest score given. The criteria were as follows: (1) Domain coverage -this set aims to be of minimal burden and complexity. Thus, tools that cover many domains were preferable; (2) Psychometric properties -the data collected must be accurate, and thus patient-reported tools were prioritized based on psychometric properties; (3) Feasibility to implement -the tool must be practical for day-to-day use in the clinic; and (4) Clinical interpretabilityclinical teams must be able to understand the results. This formed the basis for the second teleconference discussion. Finally, we sought to reach agreement on the frequency of data collection, balancing comprehensiveness, practicalities for clinics, and what would be best for patients. This was followed by identification of the baseline case-mix variables, which are necessary to make meaningful comparisons between patients. Case-mix variables to measure were prioritized based on three criteria. Each criterion was rated on a Likert scale of 1-4, where one was the lowest score and four was the highest score given. The criteria were as follows: (1) Relevance (strength of association between the case-mix variable and the outcome) -we aimed to identify case-mix variables that could strongly affect the outcome; (2) Case-mix variable independency -given the aim to collect a minimum set of case-mix variables, the aim was to identify variables that would independently affect the outcome; (3) Feasibility to collect -the set must be practical for use in the clinic. This formed the basis for the third teleconference discussion.

Literature search strategy
The following PubMed MeSH terms and Boolean logic were used to perform a search to identify outcomes that matter to PD patients, as well as scales to collect those outcomes: ("Parkinson's disease" OR "Parkinson disease" OR "Parkinsonism") AND ("critique" OR "recommendation" OR "review") AND ("scale" OR "scales" OR "instrument" OR "instruments" OR "questionnaire" OR "questionnaires"). Limitations were applied, which included the need to be review articles, written in the English language, and published in the 10 years preceding January 2015.
From this search, article titles and abstracts were reviewed to identify those that had a clear focus on scales used in clinical practice. From these results, references to scales were extracted and through targeted searches, original validation studies and use of the respective instruments were identified.

Scope
The set was designed to cover all cases of adult (>18 years of age) idiopathic PD. Atypical parkinsonism was excluded, as the consensus was that this would require different outcome measures. We recommend that atypical causes of parkinsonism be considered in future outcome sets. This set is intended to be relevant to PD patients receiving all common treatment options for motor and non-motor symptoms, including pharmacotherapy (including infusion or injection-based delivery), deep brain stimulation, and rehabilitationbased therapy (including allied health interventions, nursing, and behavioral therapy).

Outcomes
A series of motor, non-motor and other outcomes were agreed upon by the Working Group as essential to collect.

Non-motor symptoms
Non-motor outcomes impact the ability of patients with PD to carry out normal day-to-day activities [12] and are key determinants of their perceived health [13,14]. Based on the current literature, non-motor symptoms that are most important for PD patients were listed [6,7,11,15,16]. As described in Methods, the project team then prioritized this list and suggested the following outcome domains for inclusion in the standard set: depression, anxiety, cognitive function, urinary function, gastrointestinal function, pain, sleep, sexual function, treatment complications (hemorrhage and behavior change). These were deemed frequent, of high impact on patients, treatable and feasible to capture in clinical practice. During the teleconference, the group agreed with their inclusion but additionally felt that fatigue, hallucinations and sweating should also be included, due to their impact on patients. In the survey following the teleconference, the voting confirmed inclusion of the aforementioned outcomes with the exception of treatment complications -specifically, hemorrhage, as it is very uncommon, and behavior change, as this is captured under the cognitive and psychiatric domains. Additionally, the survey revealed that psychosis, apathy, impulse control disorder and dizziness/syncope were further domains deemed necessary to be part of the standard set, again due to their impact on patients. These were reviewed at the next teleconference and agreed by all WG members to be included in the Set. (See Table 2 for the full list of outcome domains and suggested scales).
It was felt that it would be simpler and less burdensome for patients and health systems to have a single instrument rather than many individual patient-reported outcome measurements. A number of scales were considered, including NMSS [19], NMSQuest [19,30], SCOPA-AUT [17] and MDS-UPDRS Part 1 [31,32]. Ultimately, the MDS-UPDRS part 1 was chosen, as it has the highest test-retest reliability and internal consistency (as measured by Cronbach's alpha), in comparison to the other tools, as well as having acceptable construct validity. Additionally, it poses minimal burden on the health system, with the clinician-recorded component taking <10 minutes to complete and the rest being patient reported [32]. Additionally, the MDS-UPDRS Part 2 (see below) is recommended for collection of the motor outcomes and thus it was felt simpler for clinics to use the MDS-UPDRS for both motor and non-motor assessment.
Two of the selected domains (sweating and sexual function) are not covered in the MDS-UPDRS part 1 survey, so it was decided to use the questions addressing these issues that are in the NMSQuest [21]. While not a perfect solution, the Working Group prioritized the selection of two simple, easy to administer, patient-reported questions. The Working Group encourages the MDS to consider including questions relating to sweating and sexual dysfunction in future iterations of the MDS-UPDRS.
We initially considered using the MDS-UPDRS part 1 as a screening tool for anxiety, depression and cognitive symptoms, and to use domain specific scales such as Beck Depression Inventory (depression) [33], State Trait Anxiety Inventory (anxiety) [34] and Montreal Cognitive Assessment (cognition) [35] to investigate these non-motor symptoms in more detail. However, it was decided that this would miss a key principle underpinning the work (i.e., to produce a practical, minimum set of outcomes that is of minimal burden to patients and staff). Therefore, only the MDS-UPDRS part 1 was included as part of the set.

Motor symptoms
Motor symptoms are an important problem in PD and their presence is relied upon to make a clinical diagnosis of PD. Motor features that were considered to be most important to the PD patient were identified and listed [6,7,11]. The outcome domains that the project team suggested including in the standard set (following the process set out under the methods) included: mobility -ability to walk; activities of daily living -living independently, handwriting and keyboard capabilities; ability to self-care; tremor; speech; swallowing; treatment complications (dyskinesia and dystonia). During the teleconference (and confirmed by the post-teleconference survey) it was agreed to include these proposed outcome domains, and it was suggested and agreed upon in the post-call survey to include additional ones. The additional outcomes included: leisure activities, saliva and drooling, and ability to move in bed at night. These were agreed upon as they are domains that can have a significant impact on the patient's quality of life. Ultimately, the only outcome domains from the initial list not to be included in the standard set were treatment complications -specifically, dyskinesia and dystonia -as it was felt that we should focus on motor function, not specific symptoms or side effects.
A wide variety of rating instruments were identified for different motor symptoms, including the Hoehn and Yahr staging [36,37], the Schwab and England ADL scale [38], PD-related health status questionnaires [39] such as PDQ39 [40], the MDS-UPDRS, and scales which can be used to report motor complications, such as "wearing off" [41], risk of falling (including the Berg Balance Scale [42] and others [43,44]) and mobility (Timed Get Up and Go Test) [45]. During the teleconference discussions it was agreed that many domainspecific scales would be needed and that this would be too burdensome and complicated for patients and clinical teams. Therefore, the MDS-UPDRS and the PDQ-39 were ultimately identified as the potential tools for data collection. The PDQ-39 is available in multiple languages and is free to use, but only covers 6/10 motor domains that we identified as being important. In contrast, the MDS-UPDRS part 2 questionnaire is also available in multiple languages and is free to use clinically but covers 10/10 domains. MDS-UPDRS part 2 has excellent psychometric properties [20]. Therefore, the MDS-UPDRS part 2 was decided as the motor tool of choice by the Working Group.

Additional health outcomes
We identified four additional domains as important for patients with PD: ability to work, hospital admissions, overall PD-related health status, and falls. These were selected by the group, particularly the patient representative, as important outcomes to assess. To assess ability to work, hospital admissions, and falls, the questions currently used in the recently developed Dutch National Parkinson's Disease Registry (www.ParkinsonInzicht.nl), which cover these domains, were selected for use in the ICHOM set. The Dutch registry uses the PDQ-39 to assess PD-related health status. The PDQ-8 and PDQ-39 are comparable as health status indices, but the PDQ-8 is significantly less burdensome to complete [46][47][48].
We recognize the value of having a single PD-related health status score and decided to include the PDQ-8.
Finally, there was also a discussion around the assessment of cost of accessing care for the patient. While we agreed that cost is vitally important, it was best included not as an outcome but rather the denominator of the value equation. Reporting cost was therefore seen as out of the scope of this work.

Case-mix variables
Patients with PD have a broad range of characteristics both related and unrelated to their neurodegenerative disease that may influence their outcomes. A parsimonious set of casemix variables ( Table 2) that were felt to strongly impact outcomes, based on existing de Roos et al.
literature [49,50] and informal discussions, was proposed. For demographic variables: age, gender, level of education, and living status (i.e. whether the patient was living alone) were proposed. Age and gender are associated with anxiety, cognitive function, urinary function, GI function, pain, sexual function and fatigue. Gender is associated with depression [51]. Level of education, gender and living status are associated with cognitive function [49,52]. For baseline health status: early age at onset of PD, depression earlier in life, PD motor subtype, non-PD related cognitive dysfunction, non-PD related co-morbidities, and non-PD related medication affecting sleep, sexual function, and dizziness were proposed. During the teleconference it was suggested and agreed upon to include marital status as an additional demographic variable, as not being married is known to be associated with the risk for cognitive decline in the elderly general population [53]. Other constructs such as loneliness and social networks in late life also include marital status and are known to be correlated to cognitive function [50]. There was unanimous agreement to remove PD motor subtype and all medication side effects due to the difficulty of recording this information accurately.
There was agreement to change early age at diagnosis of PD to age at diagnosis of PD, as there are conflicting views on the definition of "early", while age would provide a more specific time point assuring less ambiguity in the data collected. For baseline health status, the age of PD onset and diagnosis, the diagnosis of depression, anxiety or rapid eye movement (REM) sleep behavior disorder (RBD) before PD diagnosis [53], and comorbidities were included. We agreed on definitions for each of the case-mix variables.
For marital status and living status we decided to use the widely accepted definitions developed by the European Social Survey [54]. For level of education, the United Nations Educational, Scientific and Cultural Organization (UNESCO) definitions of education levels, which allow for international and cross-cultural comparisons, were selected [55]. We decided to change the term "tertiary" to "University or equivalent" as it was felt that this wording would be easier for patients and care providers to understand. For the case-mix variables, depression and anxiety, we developed two new yes/no questions. We agreed to include a single baseline patient-reported question used to assess previous REM sleep behavior disorder [53]. A validated patient-reported Charlson Comorbidity Index currently in use by the United Kingdom National Health Service [56] was chosen to reduce data collection burden on physicians.

Data collection
In order to be able to easily compare between providers, centers and countries, the use of established instruments with multiple translations was prioritized and data collection methods that can be applied across different countries and settings were proposed. We aimed to reduce the reporting burden on clinicians and as such the vast majority of outcomes in the set are patient-reported, with the exception of the cognitive and mental health outcomes. We recommend all outcomes to be recorded annually.
included to enable case mix adjustment so that inter-center and international comparisons can be performed. It aims to build on existing outcome measurement work [6][7][8][9][10] and additionally brings the perspective of leading clinicians and a patient advocate from around the world to ensure a global perspective.
The aim was parsimony, so more detailed symptom-specific scales (e.g., the Beck Depression Inventory and the Montreal Cognitive Assessment) were not selected. Additionally, not all possible outcome domains were included, but rather a focus on those essential outcomes that really reflect what matters to most people with Parkinson's disease in most places. For example, driving is key component of the patient's independence, and a frequently volunteered priority in clinical practice [57]. The fact that driving was not mentioned suggests that not all elements that matter to patients came to light in this project, and consequently did not make it to the final instrument. We therefore encourage teams to use this dataset as the basis on which other outcome domains can be added.
Ultimately, the MDS-UPDRS parts 1 and 2, three questions from the NMSQuest, the PDQ-8, and six questions from the Dutch National PD registry were chosen, as their questions represent all of the domains that the Working Group identified as being important. We realize that some health care providers currently use different scales and that there may be challenges in switching to the present recommendation, but we feel that the prospective benefit of being able to perform cross-provider comparisons and to collaboratively learn and improve patient care will encourage universal adoption of this set over time. We also recognize that computer-adaptive patient-reported outcome measures are currently under investigation, and that they may eventually replace the scales included in this set. To ensure continuity of the set over time, a subset of Working Group members has formed a Steering Committee to review and update the set on an annual basis.
This set aims to be used on a day-to-day basis in the clinic, as a useful tool to help guide management decisions for clinicians and patients. It is also hoped that it will be used to compare the quality of care provided by different centers around the world, stimulating discussion and learning from those centers with the best outcomes. For the MDS-UPDRS, the NMSQuest and the questions from the Dutch registry, it is envisaged that the results of each individual question will be the unit of comparison. For the PDQ-8, an overall score can be calculated, which will be used for comparison.
We are recommending existing validated instruments, and as such this dataset can be used immediately by teams across the world in pilot experiments. Specifically, before this ICHOM approach to outcome measurement can be recommended fully to international communities of clinicians, we recommend that pilot experiments should be performed in a cohort of individuals with PD. The results of such pilot studies should be evaluated using established psychometric approaches to further optimize the question set. Accordingly, we actively seek such feedback from teams to ensure that the set remains practical and relevant for people living with Parkinson's disease. For most institutions, implementation into routine clinical practice may be challenging, not in the least because it may require new resource commitments and infrastructure development. ICHOM has developed an expert implementation team to assist institutions in figuring out how to overcome these challenges. While we recognize the challenges, we are encouraged by the increasing availability of electronic health records and communication technologies that enable outcome reporting directly into the patient's medical record. We hope that this set will further spur development in this area. We also recognize that in some languages, validated translations of the proposed scales do not yet exist and will need to be undertaken. Finally, we note that valid comparisons of outcomes across countries are in their infancy and will require further methodological development to ensure validity [58].
A methodological draw back to the project was the absence of physiotherapy and rehabilitation expertise in the Working Group, as well as absence of representation from Asia, Oceania and South America. This will be addressed by identifying appropriate expertise to join the steering committee, which is charged with monitoring and updating the set on an ongoing basis.
In summary, we have developed a simple, relatively easy to implement, set of outcome indices that we believe should, after piloting testing, be collected and tracked for all patients with PD. This is an initial step towards driving meaningful and significant improvements in the care of patients with PD around the world.