{'fraction_non_alphanumeric': 0.03303834808259587, 'fraction_numerical': 0.020775389801938476, 'mean_word_length': 4.788048780487805, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Establishing minimum clinically important difference values for the Patient-Reported Outcomes Measurement Information System Physical Function, hip disability and osteoarthritis outcome score for joint reconstruction, and knee injury and osteoarthritis outcome score for joint reconstruction in orthopaedics

AIM To establish minimum clinically important difference (MCID) for measurements in an orthopaedic patient population with joint disorders. METHODS Adult patients aged 18 years and older seeking care for joint conditions at an orthopaedic clinic took the Patient-Reported Outcomes Measurement Information System Physical Function (PROMIS® PF) computerized adaptive test (CAT), hip disability and osteoarthritis outcome score for joint reconstruction (HOOS JR), and the knee injury and osteoarthritis outcome score for joint reconstruction (KOOS JR) from February 2014 to April 2017. MCIDs were calculated using anchor-based and distribution-based methods. Patient reports of meaningful change in function since their first clinic encounter were used as an anchor. RESULTS There were 2226 patients who participated with a mean age of 61.16 (SD = 12.84) years, 41.6% male, and 89.7% Caucasian. Mean change ranged from 7.29 to 8.41 for the PROMIS® PF CAT, from 14.81 to 19.68 for the HOOS JR, and from 14.51 to 18.85 for the KOOS JR. ROC cut-offs ranged from 1.97-8.18 for the PF CAT, 6.33-43.36 for the HOOS JR, and 2.21-8.16 for the KOOS JR. Distribution-based methods estimated MCID values ranging from 2.45 to 21.55 for the PROMIS® PF CAT; from 3.90 to 43.61 for the HOOS JR, and from 3.98 to 40.67 for the KOOS JR. The median MCID value in the range was similar to the mean change score for each measure and was 7.9 for the PF CAT, 18.0 for the HOOS JR, and 15.1 for the KOOS JR. CONCLUSION This is the first comprehensive study providing a wide range of MCIDs for the PROMIS® PF, HOOS JR, and KOOS JR in orthopaedic patients with joint ailments.


INTRODUCTION
The National Institutes of Health sponsored an initiative to develop a Patient-Reported Outcomes Measurement Information System (PROMIS ® ) [1] using item response theory (IRT) methods. These methods have been recommended for use in the evaluation of arthroplasty outcomes because of their unique measurement properties [2] . Computerized adaptive testing (CAT) using IRT minimizes respondent burden without sacrificing instrument precision [3,4] . The PROMIS ® Physical Function (PF) CAT assesses physical function in five domains but is not directly targeted at joint function. Yet the useful measurement and administration qualities of the PROMIS ® PF CAT make it a valuable addition to patient-reported outcomes (PRO) assessments in joint reconstruction [5,6] . Aside from minimizing burden, top quality PRO instruments also offer reliable and valid scores that are easy to interpret [7] . Recent attention in PRO development has focused in on the interpretability of scores, particularly in terms of how meaningful the outcomes are to patients [8] . Change in function is an important clinical outcome, thus PROs should be able to detect change in patient function. In interpreting change, the minimum clinically important difference (MCID) reflects the smallest amount of meaningful change [9] . "MCIDs are patient derived scores that reflect changes in a clinical intervention that are meaningful for the patient" [10] . Meaningful change is important, as it can serve as a benchmark of treatment effect, and it is critical in decision making. Whether or not a treatment produces a statistically significant outcome is less informative than whether it produces meaningful change, as an inflated sample size can yield statistical significance without clinical relevance [11,12] . Multiple methods have been developed for determining MCID values and there is little agreement on the best standard to apply [10] . Distribution-based approaches rely on statistical methods and probability sampling. They describe how much change falls beyond random levels of variation. They rely on distributions of scores and how much the scores vary between patients in reaching a magnitude of change that is beyond chance fluctuation [13] . But distribution methods cannot tell us whether the amount of change is meaningful from the clinician's or the patient's perspective [8] . An alternative approach is to use anchor-based methods which relates the change in patient scores to some other measure of health outcomes [13] .
Determining the MCID of the PROMIS ® PF is an important step to understand the meaning of the scores. Collecting the longitudinal data necessary to analyze meaningful change takes time. Because the PROMIS ® development began quite recently in 2004 [1] , there have been very few studies estimating MCIDs for PROMIS ® measures and little is known about MCID values in the orthopaedic adult reconstruction population [14] . Initial MCID development for PROMIS ® instruments has begun in specific patient populations such as pediatrics [15] or cancer patients [16] but studies are lacking in orthopaedic patients.
Other PROs have been developed that are specific to the domain of joint function and it is helpful to consider the measurement properties of these newer instruments side by side. The knee injury and osteoarthritis outcome score for joint reconstruction (KOOS JR) and the hip disability and osteoarthritis outcome score for joint reconstruction (HOOS JR) are two joint specific instruments recently introduced in the arthroplasty arena [17][18][19][20] . Both instruments have been approved by the Centers for Medicaid and Medicare Services for use in joint replacement registries [21][22][23][24] . The HOOS and KOOS were selected given their use in joint replacement registries in the United States, though other valid measures such as the Oxford hip or knee score are more common in European registries and the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) has a longer-term use for more general assessments of osteoarthritis [21] . Prior research has demonstrated that the HOOS and KOOS measures are sensitive to change [25] ; however, we are unaware of any studies that develop MCIDs for these instruments [26] . The purpose of this study was to determine the MCIDs of the PROMIS ® PF CAT, the HOOS JR, and the KOOS JR in an orthopaedic population with lower-extremity joint conditions.

Patient sample
After receiving approval from the University of Utah Institutional Review Board, we analyzed data from an adult reconstruction population at a single academic medical center. All patients, aged 18 and older with pathology of the hip and knee were eligible for inclusion. As part of the standard of care, patients were provided with internet enabled hand-held tablets to answer demographic and PRO instruments which are linked to the electronic health record system via mEval. Questionnaires were either completed at clinic checkin or within 7-d prior to their visit via email. Follow-up data were obtained in the same manner at future clinic visits. No informed consent was obtained as all PRO measurement was conducted as a part of customary standard of care. For this study, these visits were organized into four follow-up periods based upon the first recorded score within the database (baseline initial assessment): (1) 3-mo follow-up (80 to 100 d after initial assessment); (2) > 3-mo follow-up (90 d or more after initial assessment); 6-mo follow-up (170 to 190 d after initial assessment); and (3) > 6-mo follow-up (180 d or more after initial assessment). These followup periods were selected based upon recommendations within the literature [27][28][29][30][31][32][33][34][35][36][37] . It should be noted that the baseline score may not necessarily correlate with a specific intervention. Nonetheless, this method still allows for the monitoring of change over time.
A major goal of MCID determination is to allow for meaningful interpretation of scores for clinical decision-making, in addition to sample size calculation for investigating treatment effectiveness. As there is essentially no evidence that MCID values are dependent on the severity of disease conditions, length of followups, or specific patient groups [38][39][40] , it is appropriate for this study to establish MCIDs among orthopaedic patients with a full range of joint impairments and varying follow-up time points. The present evaluation of MCID values was conducted in a general joint clinic population, among surgical and non-surgical patient samples regardless of specific treatment or intervention. Since MCID development were not meant to be treatment specific [40] , MCID values derived from this study can be applied to adult reconstruction patients with all types of surgical and non-surgical interventions.

Patient-reported outcomes
The PROMIS ® PF CAT, v1.2, draws from a 121-item test bank that contains both upper extremity and lower extremity functional items. The PROMIS ® PF CAT algorithms were established by PROMIS ® developers [41] , and the instrument was scored using T-scores, a standardized metric that has a mean of 50 and a standard deviation of 10 [5] . Higher scores on the PROMIS ® PF CAT indicate higher physical function.

HOOS JR:
The HOOS JR is a 6-item measure assessing function and pain [22] with psychometric properties similar to the full HOOS [42] . The HOOS JR was scored on a 0 -100 scale with larger numbers indicating higher hip function.

KOOS JR:
The KOOS JR is a 7-item measure assessing function and pain [24] . The KOOS JR is scored on a 0-100 scale with larger numbers indicating higher levels of knee function.

Statistical analysis
Descriptive statistics regarding patient characteristics and demographics were calculated. Mean change scores for the patients were evaluated for each timeperiod. Change scores were calculated as the follow-up score minus the baseline score on each measure, and recorded as the absolute value difference between the scores. They were calculated for each of the follow-up periods described above including 3-mo, > 3 mo, 6-mo, and > 6 mo time-points.
The anchor-based methods applied patients' perspective to the question: "Compared to your FIRST EVALUATION at the University Orthopaedic Center: how would you describe your physical function now?" (much worse, worse, slightly worse, no change, slightly improved, improved, much improved) as a determinant of meaningful change. No change equates to a 0 value; the negative ratings are from -3 to -1 and positive ratings are from 1 to 3. When change is anchored to the patient perception or report of deterioration or improvement, it can be interpreted as a meaningful (or noticeable) level of change [9] . Patients with a ± 2 or ± 3 point change (much worse, worse, improved, much improved) were included in each analysis of change, a method used to distinguish noticeable change from nochange [43,44] . Patients reporting no change or only slight change were considered together as the no-change group. We combined the improved and deteriorated conditions using absolute values of the change scores to distinguish change from stable symptomology [45] . The distribution-based methods included calculations based on the standard deviation (SD) and on the minimum detectable change (MDC). The SD approach used the 1/2 SD and 1/3 SD as variation-based estimates of MCID. The MDC is based on the standard error of measurement (SEM) of the follow-up scores and is the smallest score change that likely reflects a true change in condition. We calculated MDC at three confidence levels: 90%, 95% and 99%. , where r is the reliability represented by Cronbach alpha and SD is the standard deviation of the follow-up scores.
We fitted the receiver operating curve (ROC) to measure the best cut-off points to maximize sensitivity and specificity of the instruments. The cut-off was calculated as (sensitivity + specificity) -1, based on Youden's J value [46] . Sensitivity is the proportion of correct identification of patients who showed changes, and specificity is the proportion of correct identification of patients showing no meaningful change. All statistical analyses were performed using SPSS 24.0 (IBM SPSS Statistics for Windows, Armonk, NY: IBM Corp.) [47] , and R 3.30 (R Development Core Team, Vienna, AT: R Foundation for Statistical Computing) [48] .

Demographics
A total of 2226 patients were included in the study. Mean age of participants was 61.16 years (SD = 12.84), with a range of 18-93 years. Most participants in the study were White or Caucasian (89.7%) and Non-Hispanic (93.2%). Detailed demographics can be found in Table 1.

Anchor-based methods
Mean change: Mean change scores varied at each follow-up period. In terms of patients experiencing change, the highest mean change scores were observed at the 6-mo follow-up for the PF CAT (8.41), HOOS JR (19.68), and KOOS JR (18.85). The lowest mean change scores for patients experiencing change was at the 3-mo follow-up for all three measures. For the PF CAT mean change at 3-mo was 7.60 points, for the HOOS JR it was 14.81 points, and for the KOOS JR. It was 14.51 points. The median MCID value from the complete range of all two anchor-based methods and five distribution-based methods analyzed at all four time-points was similar to the mean change score for each measure and was 7.9 for the PF CAT, 18.0 for the HOOS JR, and 15.1 for the KOOS JR ( Figure 1).

ROC curve:
The ROC area under the curve was used to identify the optimal cut-off point between meaningful and non-meaningful change, by calculating the point at which the sum of false positive and false negative identifications are the fewest [49] . The highest ROC MCID value for the PF CAT was 8.18 at > 6-mo follow-up. For the HOOS JR the highest value was 43.36 observed at the 3-mo follow-up with a small sample size (n = 24). For the KOOS JR, the highest MCID value was 8.16 at > 3-mo follow-up. Detailed mean change and ROC cut-off values for the PF CAT, HOOS JR. and KOOS JR. can be found in Table 2.    (Table 3).

DISCUSSION
Determining the values of MCID for PRO measures enhances the interpretability and utility of the instruments. Determining score changes which reflect meaningful change requires understanding the patient perception of improvement or worsening. Anchor-based methods of determining an MCID tie the change scores to the patient reports of change and supplement the distribution-based methods which focus on precision in scores from a statistical standpoint. The current    in a pediatric population [15] or the 4-6 point PROMIS ® PF CAT MCID values identified in a cancer population [16] . The larger MCID values in this study could have been obtained for a number of reasons. The values could be population specific, reflecting an underlying difference in functional gain between the orthopaedic adult patient population and the pediatric and cancer patient populations previously studied. However, the most likely explanation is that the MCID value obtained is a reflection of the method used for calculation. Unlike other studies that commonly use just one approach to estimate MCID values, the current study used multiple approaches and multiple time points to arrive at twentyeight MCID values per instrument, allowing better triangulation of results. This study found a great deal of consistency across the follow-up time-points for any one particular method of calculating MCIDs (Tables 2  and 3), suggesting that the methods were responsible for the variation of the MCID values and that the length of follow-ups was mostly irrelevant. It is important to recognize that this study incorporated a wide array of cut-off standards, ranging from lenient to extremely strict. Since MDC95% and MDC99% are extremely strict cut-off standards, it is not surprising that these MCID values are high. None of the research reviewed as background for this study calculated MCIDs using such strict criteria [15,16] . The lowest MCID values derived from this study may be appropriate for screening purposes, but the median value (Figure 1) may be a better estimate of true and meaningful change when applying a conservative standard for evaluating treatment effects or for respondent analyses [50] . The more stringent cutoff standard derived MCID values reported in this study provide more definite assurance that important change has occurred.
The mean change for the HOOS JR ranged from 14.81 to 19.68. The ROC area under the curve ranged from 6.33 to 43.36. The 43.36 value appeared to be an outlier and is not considered a reliable estimate. The SD values ranged from 3.90 (1/3 SD) to 10.96 (½ SD). The MDC method of detecting change yielded a range of MCIDs of 17.54 (MDC90%) to 43.61 (MDC99%). These SD values are consistent with previous research identifying an MCID of 9.1 (1/2 SD method) for the HOOS in an arthroplasty population [51] . The median of the HOOS JR MCIDs was 18.0 and may be an appropriate conservative estimate for evaluating treatment effects. For the KOOS JR, the mean change ranged from 14.51 to 18.85. The ROC maximized MCID values ranged from 2.21 to 8.16. The SD values ranged from 3.98 (1/3 SD) to 8.02 (½ SD). The MDC method of detecting change yielded a range of MCIDs from 20.84 (MDC90%) to 40.67 (MDC99%). The MDC90% values reported here are actually lower than previous research on the KOOS JR which produced a range for improved individuals from 28.3 to 35.5 [52] . The median KOOS JR MCID was 15.1 in this patient population.
The findings demonstrate that the method used  to estimate MCIDs has a large impact on MCID value determination. The KOOS JR, for example, had a 2 point to 40 point difference in MCID depending on the method used. There is not yet consensus on a standardized approach for establishing MCID [10] . The lack of agreement between MCID values reported in this study, depending on the method used, is consistent with findings in the literature [53] . There have been recommendations that MCID determination should be standardized, but the best methods have not been agreed upon [38] . Until a more standardized method is established, the comprehensive range of MCIDs presented in this study provided much deeper insights than many existing studies. This comprehensive presentation enables patients, clinicians, care-givers and decision makers to be well-positioned in making their judgment call as to which MCID value(s) they should select based on how lenient or strict a standard they would like to set for their patients. It is tempting for clinicians to oversimplify and search for a single fixed MCID value, yet a single MCID value is often unstable [8] .
A range of MCID values such as those presented in this study should indeed be considered by clinicians and health care practitioners.
The study may be limited by the type of patients who self-selected to return for follow-up visits, which is a common phenomenon across all orthopaedic and other clinics. Patients returning for longer-term followup were generally those experiencing more severe conditions and thus may not reflect the full range of improvement in condition. However, this should not be of much concern, as these patients returning for followups were representative of the ones treated regularly in clinics, rendering the results of this study even more practical for standard orthopaedic practice. MCID value determination is generally not dependent on the severity of condition, thus the shorter-term or longerterm follow-up periods would not have impacted the MCID values, as evidenced from the empirical results of this study. In addition, not every patient in this study completed the outcome measures pre-treatment, thus some change scores may reflect a baseline time-point that was post-intervention. Yet these evaluations still produced meaningful change since longitudinal change over time was the main focus, not necessarily change from a specific time point to another specific time point.
One additional limitation comes from the use of anchor-based questions to determine clinically relevant change. An anchor-based question with a global rating of change measure is subject to recall bias, with some research indicating that reports of change may be more related to the current health status than real change from baseline [10] . Different anchor questions may produce different results as well. Distribution methods also have limitations as they derive MCIDs based on the variance of the data, which might be difficult to interpret. Since all methods have strengths and limitations, it was thus our intention for this study to be comprehensive in nature to cover a variety of approaches, allowing readers to make informed decisions in selecting MCID values based on their personal value judgments as to which target MCID is worthwhile to pursue. Lastly, the demographic characteristics of this sample may not be representative of those in the United States and may affect MCID scores. Future research should investigate MCIDs in a more diverse demographic sample. It should also establish MCIDs linking baseline scores using Rasch methodology to provide deeper insights.
Overall, this study utilized rigorous methodologies to develop a wide range of MCID values for the PROMIS ® PF, HOOS JR, and KOOS JR in an orthopaedic sample of patients with joint disorders. As there is no such concept as a correct or incorrect MCID, individual value judgments are necessary to apply MCIDs to treatment planning and in guiding patient expectations of treatment change.

Research background
Newly developed patient-reported outcomes have many advantages, but require further studies, including establishing minimum clinically important difference (MCID) values. Determining the MCID for the Patient-Reported Outcomes Measurement Information System Physical Function (PROMIS ® PF) will be useful for orthopaedic clinical practice and it is helpful to understand the MCID in the context of previously used measures like the hip disability and osteoarthritis outcome score for joint reconstruction (HOOS JR), and the knee injury and osteoarthritis outcome score for joint reconstruction (KOOS JR), Anchor-based and distribution-based methods can both be used to determine MCID.

Research motivation
New instruments require studies to inform their score interpretation. Because of the lack of consensus on MCID methods, a comprehensive approach was taken, using both anchor-and distribution-based methods at multiple levels of precision and multiple follow-up time points. Cross verification of MCID values using powerful triangulation methods allow researchers and clinicians to understand the complexity of MCID evaluation and conscientiously select the most appropriate one for themselves.

Research objectives
To determine MCIDs for the PROMIS PF, HOOS JR and KOOS JR in a general joint orthopaedic patient population applying comprehensive methods.

Research methods
Consecutively enrolled patients aged 18 and older from a large academic orthopaedic joint clinic completed PROs at their first clinic visit and at followup points from 3-mo to 6-mo and beyond. These patients also completed an anchor question that queried how much their physical function had improved since their first clinic visit. They were grouped into change and no-change categories. Anchor-based analyses looked at mean change scores and the receiver operating curve to maximize the best cut-off based on sensitivity and specificity. Distribution-based analyses looked at the standard deviation, and minimum detectable change.

Research results
There were 2226 patients who participated with a mean age of 61. 16