{'fraction_non_alphanumeric': 0.024444274692536856, 'fraction_numerical': 0.008479107783973722, 'mean_word_length': 5.341244853475418, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Developing a Standard Set of Patient-centred Outcomes for Adult Oral Health – An International, Cross-disciplinary Consensus

Objective To develop a minimum Adult Oral Health Standard Set (AOHSS) for use in clinical practice, research, advocacy and population health. Materials and methods An international oral health working group (OHWG) was established, of patient advocates, researchers, clinicians and public health experts to develop an AOHSS. PubMed was searched for oral health clinical and patient-reported measures and case-mix variables related to caries and periodontal disease. The selected patient-reported outcome measures focused on general oral health, and oral health-related quality of life tools. A consensus was reached via Delphi with parallel consultation of subject matter content experts. Finally, comments and input were elicited from oral health stakeholders globally, including patients/consumers. Results The literature search yielded 1,453 results. After inclusion/exclusion criteria, 959 abstracts generated potential outcomes and case-mix variables. Delphi rounds resulted in a consensus-based selection of 80 individual items capturing 31 outcome and case-mix concepts. Global reviews generated 347 responses from 87 countries, and the patient/consumer validation survey elicited 129 responses. This AOHSS includes 25 items directed towards patients (including demographics, the impact of their oral health on oral function, a record of pain and oral hygiene practices, and financial implications of care) and items for clinicians to complete, including medical history, a record of caries and periodontal disease activity, and types of dental treatment delivered. Conclusion In conclusion, utilising a robust methodology, a standardised core set of oral health outcome measures for adults, with a particular emphasis on caries and periodontal disease, was developed.


Introduction
Oral diseases directly impact the lives of individuals by causing considerable pain and suffering, altering food choices, affecting speech, self-esteem, quality of life, and participation in everyday activities 1,2 . Measuring the impact of oral diseases has traditionally been based upon the biomedical model that provides only limited insight into the impact of oral disease on people's lives 3 . The emerging patient-centred care model 4 necessitates a focus on oral health rather than oral disease. Although measures of oral health and oral health-related quality of life have been developed, they have not been shown to be useful in all of the important domains of clinical practice, health services research, epidemiology and advocacy. This is reflected in recent definitions of oral health that now include physical, psychological, emotional and social domains, which are core to overall health and wellbeing 5−7 . From the patient-centred care perspective, oral healthcare providers should thus consider not only disease processes, but also the environmental, social and personal factors, overall quality of life and participation in all major life areas, including making decisions about and control over their health and the use of health services 8 . This approach is reflected in the FDI World Dental Federation (FDI) definition of oral health 6 (Figure 1), and provides a theoretical framework for shared decision-making in clinical practice, as well as for health services research, epidemiology and advocacy.
The ability to measure oral health outcomes from both the clinician and patient/consumer perspective in a simple reproducible manner is fundamental to the principle of valuebased oral healthcare. This takes into account the achievement of health outcomes that matter to patients and the healthcare costs incurred in achieving those 9−11 , and is a central premise of the International Consortium for Health Outcomes Measurement (ICHOM). Defining a minimum standard or core set of health outcome measures that are meaningful to both patients and clinicians is a necessary first step towards assessment of the value of care provided. Accordingly, ICHOM has thus far developed 27 core outcome standard sets for a range of human diseases and conditions. Each standard set developed by ICHOM has the following components: (i) patient-centred outcomes; (ii) case-mix variables (which are factors that will affect the patient-centred outcomes that are not influenced by the management of the condition and are used to allow the construction of riskadjustment models); (iii) validated instruments that can be adult oral health standard set used for the outcomes and case-mix variables; (iv) clinicianor patient-reported data sources; (v) specific time-points for data collection. Based on case studies of the implementation of these standard sets in clinical care, the benefits included fewer delays in clinical practice, greater efficiency in administrative processes and consultations with a greater focus on what mattered most to the patients. Patients reported that they were more engaged in their own care, it 'gives us more of a role that makes the appointment more tailored to what is important to us'; while clinicians reported that the routine use of these standard sets resulted in the consultations being 'far more focussed' with a better structure to their patient inter-actions 12 .
The key objective of the research reported here is the development of a minimum Adult Oral Health Standard Set (AOHSS) of outcome measures, produced through a collaboration between the FDI and ICHOM, for use in routine clinical practice, research, advocacy and population health. Reflecting their predominance in routine clinical practice, caries and periodontal disease are the principal focus among clinical conditions for this standard set.

Methods
An international Oral Health Working Group (OHWG) of 22 members comprised of patient advocates, researchers, clinicians, policymakers and public health experts, representing 10 countries, was established to ensure diversity in subject matter expertise, sex and geographic location (Table 1, OHWG membership). This membership unified the discipline-specific expertise of FDI with the strong methodological expertise of ICHOM.
The study included the following phases.

Phase 1: Selection of measures for AOHSS
According to the WHO, caries and periodontal diseases are considered the most important global oral health burdens 13 . The AOHSS focussed therefore on caries and periodontal disease in adults. A literature search was undertaken in PubMed to identify relevant oral health outcomes and case-mix variables published in the previous 10 years. Case-mix variables are used in value-based health to provide a contextual picture of a patient cohort. The search terms and inclusion/exclusion criteria are presented in Table 2. Duplicate articles were excluded. Two reviewers, who had been calibrated to ensure consistency, reviewed all remaining titles and abstracts independently. After reviewing the first 50 abstracts, a Cohen's kappa coefficient was calculated to determine inter-rater agreement on the application of the inclusion and exclusion criteria. This iterative process was repeated until a kappa of > 0.7 was achieved; the remaining abstracts were then distributed between the two reviewers to determine inclusion and subsequent data extraction. Supplemental literature, including public health surveys and clinical guidelines, were also reviewed.
Patient-Reported Outcome Measurement (PROM) instruments, including the Oral Health Impact Profile-14 (OHIP-14) 14 General Oral Health Assessment Index (GOHAI) 18 and OHQoL-UK 19 , were reviewed by the OHWG with respect to their coverage of patient-reported outcomes, psychometric properties, content, validity and reliability, availability of language translations, and applicability of copyright/licensing fees. A patient advisory group was organised in the USA to record response frequency of previously identified domain concepts, as well as take notes on newly raised concepts in real time. This patient insight was gathered via a focus group held in a non-clinical setting in Boston Massachusetts, USA. Eight patients, with fluency in English and who had undergone dental treatment in the last year, participated in the focus group moderated by an experienced qualitative researcher with no expertise in oral health ( Figure 2, Patient advisory group demographics).

Phase 2: ICHOM consensus-driven modified Delphi technique to develop the AOHSS
A structured, consensus-driven modified Delphi technique 20,21 was used that included a combination of videoconferences and surveys to select the AOHSS measures and case-mix variables ( Figure 3). Five face-to-face meetings took place in London (twice), San Francisco and Geneva (twice), and eight videoconferences were held (January −November 2017), five of which were followed by surveys to inform decisions.
Videoconferences were preceded as needed by breakout group meetings with subject matter experts on specific areas, including PROMs, clinical measures for caries and periodontal disease, case-mix, and data collection. Following initial presentation of material for decision at each videoconference ( Figure 2: Calls 1−8), OHWG members voted electronically on the importance of the proposed items. The consensus process required at least an 80% response rate from the OHWG on every survey before data could be analysed. OHWG members were instructed to vote only on items for inclusion that met all of the following criteria: were important to adult patients with caries and/or periodontal disease; represented endresults of care and not the process of care; were feasible to measure accurately, and were modifiable with quality improvement efforts. Items were ranked on a 1−9 scale: where 1 represented an item deemed non-essential; and 9 an item considered essential. Items ranked 1−3 by 80% of respondents were excluded; items ranked 7−9 by 80% of respondents were included. Items ranked 4−6 were re-voted after further discussion 22 . If the vote was still inconclusive, a third voting round was undertaken where a simple majority determined inclusion or exclusion.
To ensure that the terminology in relation to diagnosis and disease staging used in the clinical measures of disease was consistent with current concepts, subject matter experts in the fields of cariology and periodontal disease were consulted. Outcome measures for both of these were based on input from content experts involved in generating and publishing the most recent guidelines on management and classification of the respective diseases 23,24 .

Phase 3: Global open review to determine the appropriateness and feasibility of implementing the AOHSS
The worldwide open review comprised an online structured survey, in English, pertaining to the AOHSS draft. It was distributed among oral health professionals, educators, policymakers, researchers, industry representatives and national dental association members, building on the established networks of both ICHOM and FDI. Health professionals were encouraged to circulate the AOHSS draft within their networks, which included clinicians, policy advisors, payers, educators and researchers, to facilitate snowball sampling. Online platforms including Twitter and LinkedIn were used to share the survey links. The original English language version of the AOHSS was translated into German, French, Spanish and Arabic, and distributed worldwide to determine its acceptability, coverage of the minimum relevant concepts, and appropriateness. Health professionals were additionally asked to consider utility and feasibility. The findings from the global open review were used to inform decisions by the OHWG on outcomes and case-mix factors for inclusion in the final AOHSS, as well as the corresponding measures recommended to capture them.

Phase 1: Selection of measures for AOHSS
The literature search yielded 1,453 results, yielding 959 relevant outcome and case-mix abstracts. Eight patients participated in a focus group moderated by an experienced qualitative researcher with no expertise in oral health (Figure 2, Patient advisory group demographics). The focus group was digitally recorded, transcribed and anonymised. The patient advisory group endorsed the concepts identified from the literature. However, they also identified the financial impact of oral healthcare on individuals as an impact to be considered for inclusion in the AOHSS.

Phase 2: Delphi voting outcome
The first modified Delphi voting survey 1 (DVS1) consisted of three voting rounds, resulting in the inclusion of the following outcome areas from the 10 outcomes identified from literature review: health-related quality of life; overall patient/ consumer satisfaction; emotional well-being; oral healthrelated quality of life (functional and psychosocial aspects); and impact of oral health on personal relationships. An agreed definition for each outcome was then established by the OHWG in advance of mapping to established PROMs. Delphi voting survey 2 (DVS2) identified the question from the established PROM that best represented the defined outcome areas identified in DVS1. Input from the patient advisory group, specifically regarding the importance of each outcome area, was then combined with OHWG findings from DVS2. During this process, impact on personal relationships was excluded and financial impact was added.
Case-mix variables specific to patient demographics, lifestyle and general health, and caries and periodontal risk factors were voted upon. Aspects of clinical care, including baseline clinical data, treatment options and outcome of care, were also included in the clinical measures and were included in voting surveys. Examples of case-mix variables and clinical measures that were considered but not included were occupation, living status, salivary gland hypofunction, oral health literacy and household income.
Following the OHWG and subject matter expert consultations, it was decided that the clinically reported measures for outcomes pertaining to caries and periodontal disease staging be based on existing tools aligned with the International Caries Detection and Assessment System (ICDAS) 23 and the recently internationally agreed standards for measurement of periodontal conditions 24 .

Phase 3: Global open review to determine the appropriateness and feasibility of implementing the AOHSS
The worldwide open review generated 347 responses from participants in 87 countries (Figure 4). The respondents were from diverse sectors involved in the provision of oral health, including clinicians (67%), researchers (13%), educators (8%), along with policymakers, those involved in advocacy and industry (11%). Following a review of the draft version of the AOHSS, the responses were uniformly positive, and 93% of participants expressed an interest in receiving the final AOHSS; 82% also indicated that they would implement the set in their day-to-day practice.

Phase 4: Patient/consumer survey to determine appropriateness of AOHSS from patient/consumer perspective
The patient/consumer surveys generated 129 responses; 69% of respondents were female and 72% were aged between 36 and 65 years. Approximately 80% of patient respondents had received routine dental care, and for the overwhelming majority this had been a positive experience; 87% of respondents wanted their dentists to routinely use the AOHSS.
Patients/consumers from USA and Australia in Phase 3 and health professionals who participated in the worldwide open in Phase 4 review expressed similar views about simplifying the question language; revising the type of questions and their response options, and clarifying the concepts captured. Health professionals were concerned about the apparent exclusion of specific population groups such as those in rural areas or those with special needs. Patients/consumers cited a high literacy requirement and the length of the questionnaire as points of concern. Some questions were also perceived to be judgemental about their circumstances.

Final version of the AOHSS
After the last OHWG voting round, agreement was reached on the final version of the AOHSS. This resulted in the inclusion of 17 outcome concepts (13 patient-reported, four clinicianreported) and 14 case-mix concepts (eight patient-reported, six clinician-or administrator-reported). The 31 conceptual outcomes included in the AOHSS ultimately comprised a total of 80 measures (25-patient reported, 55-clinician/administrative reported). These 25 items were directed towards patients (including demographics, the impact of their oral health on oral function, a record of pain and oral hygiene practices, and financial implications of care) and items for clinicians to complete, including medical history, a record of caries and periodontal disease activity, and types of dental treatment delivered. A complete overview of the measures included in the AOHSS is presented in Figure 5, with Figure 6 demonstrating the mapping of the AOHSS to the FDI oral health definition framework.

Discussion
This collaboration between FDI and ICHOM used a rigorous, consensus-driven approach involving oral health experts and patients/consumers across the globe to arrive at a harmonised list of items to be included in an AOHSS. The methodology employed in the development of this outcome set followed ICHOM protocols and focused on what matters most to patients on a daily basis. This approach has resulted in the publication of 27 ICHOM sets that have been adopted in medical institutions across the globe, which are developing systems based on the principle of value-based healthcare. They range from the implementation of the ICHOM set for coronary artery disease in South Australia to the implementation of the cleft lip and palate ICHOM set at the Erasmus University Medical Centre in the Netherlands 12 . Hence, they differ in their purpose and their methodology from, for example, the core outcome sets developed in both Core Outcome Measures in Effectiveness Trials (COMET), which focuses on standardising outcome measures for clinical trials 25 , and Outcome Measures in Rheumatology (OMERACT), concentrating on rheumatological conditions 26 . In terms of value-based healthcare, there is emerging debate about the relevance of defining 'value' more widely than purely monetary in the context of cost-effectiveness, and how such concepts can be applied to oral healthcare 27,28 .
It is intended that the AOHSS will be used by clinicians and patients in a shared decision-making environment to co-produce care plans and track oral health outcome progress over time. From a clinical perspective, the focus of this AOHSS was on the most prevalent oral conditions, caries and periodontal disease, but the AOHSS can readily be adapted to include other oral diseases and conditions. The measurement of caries is broad, with the caries status of each tooth recorded as sound, restored, or with caries involving enamel, dentin or the pulp. This broad measurement can be considered as a foundational level of data collection with regard to dental caries. It does not preclude using other standard caries experience measures, such as the DMF (Decayed, Missing, Filled) classification, or even a more 'granular' level of information, such as the ICDAS classification 29,30 . Similarly, with the clinical recording of periodontal disease staging, the periodontal recording per sextant as either healthy or three grades of pocketing (< 5 mm, 5−7 mm and > 7 mm) allows an overall assessment of periodontal status that is broadly in line with more detailed assessments undertaken for periodontal patients or in oral epidemiological studies of the condition 31 . From the patient perspective, the integration of PROMs is strongly linked to quality of life measures, similar to other health disciplines. Although the potential of PROMs for shifting the pattern of care provision towards addressing the concerns and priorities of the patients themselves 32 is  recognised, their use in routine practice is still uncommon 33 , and there are a number of challenges related to their use and interpretation 34 . Although of importance, the following groups were excluded from the scope of this project, but warrant targeted follow-up work: children; special needs populations; older people; institutionalised persons; and those otherwise dependent on others for day-to-day oral care.
Reference periods for the most widely used subjective outcome measures for oral health and quality of life range between 6 and 12 months for adults, and national studies have used the latter as a time reference point. However, reference intervals in clinical studies are usually narrower in order to reflect the trajectory of changes in oral health after an intervention. The guideline from the National Institute for Health and Care Excellence in the UK recommends that recall intervals should be tailored to each patient depending on their need, and for adults should not be longer than 24 months 35 . Therefore, the recommendation on data collection time-points is flexible and could mean that, at a minimum, the AOHSS should be collected during the following stages of care: baseline, described as 0 months or prior to any new treatment; treatment and recovery, described as > 30 days after each intervention; and follow-up, described as 12 months after completion of a cycle of care or at least once every 24 months for individuals not requiring any interventions (Figure 7, Time-points). The full AOHSS does not need to be collected at each time-point. Only those data points relevant to each stage in the care cycle should be collected. Furthermore, these minimum data collection time-points should not preclude more frequent data collection, such as annually, if judged to be clinically appropriate.
Following the determination of both the validity and reliability of this newly developed standard set, it is intended that the AOHSS be used in clinical, research, evaluation and community settings. Its flexibility in data collection will facilitate cross-sectional and longitudinal use within diverse global settings. Some of the selected measures have already been included in large national epidemiological surveys to document the impact of oral conditions on daily life 36 . The applicability and adaptability of the AOHSS for the various aforementioned settings will inevitably lead to supplemental items or tools being added to this minimum standard set. As long as the integrity of the original AOHSS is maintained and the psychometric properties of this minimum set are demonstrated, following rigorous testing in a clinical, research and population health setting, the addition of subsequent variables will allow flexible data collection whilst facilitating a basic comparison. The adaptability of the AOHSS will, however, only truly become apparent following the phase of psychometric testing. At the patient level, using the AOHSS at the beginning and end of a treatment course will facilitate assessment of preventive and treatment interventions.
Although the development process for the AOHSS has been rigorous, it is not without limitations. Although the OHWG members were recruited from across the globe, there was limited representation from Asia (with one member from UAE). The initial OHWG membership included one member from Africa, who participated in the initial planning stages of the study. Unfortunately, due to the pressure of other commitments, he was unable to participate in a sufficient number of the Delphi voting rounds to meet the criteria for inclusion in the list of authors. The patients involved in its development were also not truly representative of the diverse global population for which AOHSS is intended, and were concentrated in USA and Australia. The broad scope of the data collected and the focus restricted to minimum measures for common preventable oral conditions limits the ability to collect comprehensive disease-specific information on all oral health conditions. For its effective implementation, a clear communication strategy explaining the value of data collected will be important. Clinicians will need to use these data in their clinical practice to understand what matters to patients, and share data with patients. Regular review of data generated by this set will facilitate an assessment of which interventions are more effective in achieving outcomes important to patients.
It is hoped that the FDI-ICHOM collaboration will be viewed as a model for future quality improvement in healthcare-related projects. This initiative complements other work in the field, such as the ADVOCATE project 37 . In conclusion, utilising a robust methodology and a standardised core set of oral health outcome measures for adults, with a particular emphasis on caries and periodontal disease, was developed. To inform additional refinement of the AOHSS, the next step is to conduct a validation and feasibility study in diverse clinical settings. The AOHSS will ultimately empower patients to become decision-makers and co-producers in their care, and this has the potential to result in improved patient outcomes. The AOHSS could be utilised globally and potentially facilitate international comparisons of oral health outcomes. It would also have the potential to be useful in communities with limited capacity for oral care.
Author contributions M Glick, SN Myers, DM Williams contributed to conception, design, data analysis and interpretation, drafted the manuscript, critically revised the manuscript; R Ni Riordain, SSA AlMashhadani, J Barrow, D Cole, S Hegde, E Kalenderian, WM Thomson, G Tsakos, RG Watt contributed to data analysis and interpretation, drafted the manuscript, critically revised the manuscript; K Aravamudhan, JJ Crall, JE Gallagher, J Gibson, R Kaberry, A Karki, RK Celeste, S Listl, R Niederman, T Severin, MW Smith, M Vujicic, S Whittaker contributed to data analysis and interpretation, and critically revised the manuscript. All authors gave final approval and agree to be accountable for all aspects of the work.