{'fraction_non_alphanumeric': 0.02373251090717617, 'fraction_numerical': 0.012825334737475554, 'mean_word_length': 4.979087024960648, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Registry Studies Use Inconsistent Methods to Account for Patients Lost to Follow-up, and Rates of Patients LTFU Are High

Purpose To determine methods described in the literature to account for patients lost to follow-up (LTFU) in registry studies and whether rates of patient LTFU are within acceptable margins. Methods A scoping review, where a literature search is conducted for studies from 9 arthroscopy registries, was performed on EMBASE, MEDLINE, and the annual reports of each registry. Inclusion criteria included studies with information on patient-reported outcome measures and being based on 9 national registries identified. Exclusion criteria included review articles, conference abstracts, studies not based on registry data, and studies from regional, claims-based, or multicenter registries. Studies were then divided into categories based on method of LTFU analysis used. Results Thirty-six articles were identified for the final analysis. Categories for LTFU analysis included dropout analyses (n = 10), referencing validation studies (n = 12), contacting nonresponders (n = 4), and sensitivity analyses (n = 1). Referencing validation studies was the most common method (n = 12). Majority (n = 35) of the studies exceeded the recommended maximum rates for LTFU. Conclusions Registry studies use inconsistent methods to account for patient LTFU, and rates of patients LTFU are unacceptably high. Clinical Relevance The impact of patients LTFU in studies related to arthroscopic intervention is unknown. A universal method for accounting for patient follow-up is needed.

R ecent advances over the last 2 decades have led to rapid progress in surgical techniques and innovation in implant technology. However, along with these advances there is a constant need to monitor the outcome of these surgical techniques and newer implants to ensure that adverse outcomes are identified early and intervened upon in the best interest of the patients.
The establishment of registries such as the Swedish Knee and Hip Arthroplasty registries (in 1977 and 1979, respectively), 1,2 the United Kingdom National Joint Registry in 2002 3 are methods used to achieve this level of monitoring. Registries are also useful in providing a good platform for research because of their low cost, large observational datasets, and the ease of analyzing data in comparison with other research methodologies. Outcome based registries in particular contain information on diagnosis, details of procedures, patientreported outcome measures (PROMs), and long-term follow-up of patients. After the establishment of arthroplasty registries worldwide, arthroscopy registries have also been developed.
Many such registries use PROMs in the form of a standardized questionnaire to quantify various aspects of health including pain, quality of life, and function of the joint after the procedure. PROMs are collected at multiple time points in the follow-up of patients. Unfortunately, the rates of completion of PROMs or other surveys are often low because of poor response rates or patients "lost to follow-up" (LTFU). 4 This means that studies based on these PROMs can be influenced by nonresponse bias if the patients lost to follow-up are nonrandom or "missing not at random." 5 Despite this, many studies assume their missing data is "missing at random" and subsequently assume no nonresponse bias because there is no systematic reason for patients being LTFU. 5 As a result, there is a possibility that these studies may overestimate the success of the interventions they are investigating. To avoid nonresponse bias, studies may incorporate missing data analysis to justify the validity of their data and results. Some methods for these "loss to follow-up analyses" include carrying out a comparison of baseline characteristics of patients, 6,7 referencing a validation study 8,9 that has been carried out on the registry or contacting nonresponders themselves. 10,11 However, there is no universal agreement on the methodology to be used to account for LTFU nor on the acceptable follow-up rates for studies reported from arthroscopy registries. Traditionally, an 80% follow-up is seen as an acceptable compliance threshold for a survey study to be valid, which has been attributed to Sackett at al. 12 by other studies. 13,14 However, this number was chosen arbitrarily, and there was no statistical basis for it. Alternatively, the International Society of Arthroplasty Registries PROMs Working Group has recommended a minimum follow-up rate of 60%. 15 This cutoff is also recommended by the Journal of the American Medical Association for survey-based research. 16 The purposes of this study were to determine methods described in the literature to account for patients LTFU in registry studies and whether rates of patient LTFU are within acceptable margins. We hypothesized that the rates of patient LTFU would be within the acceptable margins for survey studies and that there are methods used by authors to account for LTFU.

Methods
This scoping review was carried out using the methods outlined by Arksey and O'Malley 17 to determine statistical methods used to account for and minimize the impact of patients LTFU.

Search strategy
A computer-based literature search of EMBASE, Medline, and a manual search of the latest annual reports of each arthroscopy registry was conducted on October 22, 2020. The eligibility criteria for the included articles were determined a priori by the authors. Inclusion criteria were that studies were from the PROM collecting arthroscopy registries as outlined by Ueland et al. 14 and that the studies performed or referenced some form of LTFU analysis based either on their patient pool or on the patients in the registry the studies were based on. Studies were excluded if they did not meet these inclusion criteria, were conference abstracts or reviews, or did not have publication available in the English language.
The search strategy (Supplementary Table S1) consisted of the names of each of the included registries, (including variations of the names such as abbreviations), followed by a manual search of papers from the annual report of that registry. For example, to search for articles from the Swedish National Knee Ligament Registry (SNKLR), the search terms were "Swedish National Knee Ligament Regist*," "Swedish Knee Ligament Regist*," "Swedish National Anterior Cruciate Ligament Regist*," "Swedish Anterior Cruciate Ligament Regist*," "SNKLR," "SKLR." This was then followed up by a manual search of the 2019 annual report (latest report) of the SNKLR for additional studies.
A list of the registries included in the search are shown in Table 1. Further details on these registries can be found in Supplementary Table S2. One author carried out the search strategy, and article screening was performed by 2 authors, with any disagreements resolved by the senior author.
From the literature search, a screening was performed first of the titles and then of the abstracts. Following this, articles were selected for full-text review prior to final analysis. The references for each of the studies selected for final analysis were then scanned to identify any further relevant studies. A PRISMA (preferred reporting items for systematic reviews and metaanalyses) flowchart of the literature search is shown in Fig 1. Variables included for articles that met the inclusion criteria were the size of patient sample, percentage of patients LTFU, type of LTFU analysis performed, and the registry the study was based on. The registry annual reports that were not available in English were translated using Google Translate. The extracted data were collated in Microsoft Excel. Statistical analyses focused on descriptive statistics.

Results
In total, 283 studies were identified in the original database search with an additional 173 studies included from the manual search of annual reports. After the removal of duplicates, 233 records were identified for the screening of titles and abstracts. Following this, 157 texts were identified for full-text analysis, resulting in 36 studies included for final analysis. One thousand ninety-two references from bibliographies of the included studies were then screened, leading to 70 further full-text articles being assessed. However, none of these additional 70 studies met the inclusion criteria, suggesting the search strategy was strong enough to capture all the required articles.
Included studies were divided into 5 categories (Table 2) depending on the type of LTFU analysis that was carried out. The main themes identified in the articles to account for loss to follow-up were (1) dropout analysis, (2) reference of a validation study, (3) contacting nonresponders, and (4) sensitivity analysis. No papers from the DGOU, DART, or NZACLR met the inclusion criteria to be eligible for this review. The reported rates of LTFU from each study was also noted.
The details of the included studies, including the reported rates of LTFU, are summarized in Table 3. Following this, the maximum and minimum reported LTFU rates from the studies from each of the registries were compared to determine whether data from any 1 registry met any of the recognized follow-up percentage guidelines. These values are presented in Table 4.

LTFU Analysis among the registries
From the 36 included articles, 5 categories of LTFU analyses were derived.

Dropout analysis
Dropout analyses were one of the most common methods of LTFU analysis with 10 studies reporting on this. This involved comparison of the baseline demographic details (e.g., age and sex) between patients who completed the PROMs and those who did not. The     Some studies referenced a validation study, but unless they were referencing a validation study on the register that the study itself is based on, they did not meet criteria for category 2 and thus did not meet the inclusion criteria. most common finding was that nonresponders tended to be of male sex and younger age. 6,7,[18][19][20][21][22][23][24][25][26][27] However, some studies found worse outcomes in nonresponders such as having more ACLR revisions. 25

Referencing a validation study
Twelve papers 28-39 referenced a validation study only, and an additional 3 articles 7,19,20 referenced both a validation study and carried out a dropout analysis. Two validation studies were carried out on the dataset of 2 registries: the SNKLR and DKRR. These studies evaluated the completeness and the quality of the data in their respective registries to determine whether reliable conclusions could be drawn from them. This also included identifying whether there were any differences in demographic details or PROMs between responders and nonresponders in the registry.
Rahr-Wagner et al. 9 carried out a validation study on the DKRR. The authors sent out 100 questionnaires to a sample of both, responders and nonresponders. Comparisons were then drawn between the responses from these. It was found that nonresponders tended to be male and of a younger age. For the SNKLR, a similar strategy was used by Reinholdsson et al. 8 where all nonresponders (n ¼ 1723) were contacted, of which 349 (21%) replied. Again, it was found that nonresponders tended to be male and younger, but they also tended to have a higher level of pain and lower quality of life.

Contacting Nonresponders
Four studies contacted nonresponders to increase the follow-up compliance percentage in their study. Lind et al. 10 contacted 200 responders and 200 nonresponders, out of which 60% replied and showed that there was no difference in the Knee Injury and Osteoarthritis Outcome Score (KOOS) scores at the 1-year follow-up between the 2 groups. Maempel et al. 11 attempted to contact each nonresponder a maximum of 5 times, after which they were able to increase their compliance to 81% (LTFU of 19%). The aforementioned validation studies by Reinholdsson et al. 8 and Rahr-Wagner et al. 9 are also included in this category.

Sensitivity Analysis
From the included studies, Holleyman et al. 38 conducted a different sensitivity analysis to adjust for differences in demographics between responders and nonresponders. This was carried out by creating a cohort ("overall cohort") with bootstrapping (random sampling with replacement) responders and nonresponders, followed by creating another cohort ("responder cohort") via the same process of random sampling from all of the responders. Next, randomly selected responders from the "overall cohort" were transferred into the "responder cohort" and removed from the "overall cohort." Nearest neighbor matching was used to match cases from the remaining patients in the bootstrapped overall cohort to those in the responder cohort based on age, sex, body mass index, and surgical diagnosis. This was repeated 1000 times. This did not aim to find differences between patients who were and were not LTFU but aimed to mitigate the bias that may have arisen as a result of the differences. Rahr-Wagner et al. 39 also mention a sensitivity analysis very briefly but do not display the data or the methodology behind it and therefore is not included in this category.

Other
This category was formed of studies that did not meet the descriptions of the other categories but still performed some form of LTFU analysis. Of the six studies in this category, one was based on the DKRR and 5 on the SNKLR. The study from the DKRR looked at reasons for LTFU by focusing on the administrative failures of several hospitals when sending data to the DKRR. 40 From the 5 studies from the SNKLR, 1 only looked at differences in response rates between male and female sexes 41 and 4 stated that there were no differences between responders and nonresponders (suggesting a dropout analysis was performed) while not providing data as evidence for this. [42][43][44][45]  No studies from the NLR or the DGOU were eligible for inclusion and therefore are not included in the table.
*Only one study met the eligibility criteria from the DHAR, therefore the highest and lowest compliance is the same y Patients were contacted by survey if they did not reply to the 1-year follow-up survey. The median response time was 24.3 months

Discussion
The main findings of this study were that there was no consistent method used to account for LTFU in arthroscopy registry studies. Many studies did not have any methods to account for loss to follow-up, and from those that did, there disagreement on how to tackle it and minimal acknowledgment of the potential bias it can introduce.
Despite a number of arthroscopic outcome studies acknowledging patients being LTFU as a limitation of their work, only 36 studies included some variant of a LTFU analysis in our review. Amongst these studies, there was no consensus on which was the best method of accounting for LTFU. The use of dropout analyses (n ¼ 10), using validation studies (n ¼ 12), or a mix of both (n ¼ 3) were the most commonly used, whereas only 1 study used a sensitivity analysis.

Dropout analysis
Dropout analyses have the advantage of allowing a direct comparison of demographic details between patients who are lost to follow-up and those who are not. With such data, information on the likelihood of a patient from a particular demographic background (sex, age, preoperative clinical picture, and PROM scoring) can be used to predict whether the patient will be LTFU. From this, patients who at a higher risk of being nonresponders can then be focused upon with additional procedures of making contact such as trying to organize a phone call to ask them to be followed up, rather than an email.

Referencing a validation study
Both validation studies found that were only minimal differences in PROMs between responders and nonresponders. For the validation study on the DKRR, only 62% (62/100) of responders and 32% (32/100) of nonresponders replied to the questionnaire. Since only 32 participants represented the entire cohort of nonresponders from the registry, it would be incorrect to conclude whether factors did or did not differ between responders and nonresponders in the DKRR. 9 For the validation study on the SNKLR, despite all nonresponders being contacted for follow-up, only 21% (359/1723) of nonresponders responded to the additional questionnaire. Because the majority of nonresponders still did not provide reasons for being LTFU or return their PROMs, there may have been other differences between the nonresponder and responder group in terms of outcomes and PROMs. 8 The low response rate from nonresponders in both of these studies suggests that the noncompliance was not due to chance but due to an underlying reason. Significant factors for nonresponders included male sex and a younger in age in both studies. An increased level of pain and worse quality of life KOOS score in nonresponders was noted by Reinholdsson et al. 8 Only 2 studies 7,28 of the 8 from the SNKLR that refenced the validation studies acknowledged the difference in these KOOS scores. The other studies from the SNKLR and the DKRR had simple statements stating there were no significant differences between responders and nonresponders.
Many studies refer to these to validate their dataset despite failing to recognize the limitations of these two studies. However, without a higher sample size for both studies, it may be disingenuous to make this assertion.

Contacting Nonresponders
Contacting nonresponders has the advantage of increasing the sample size of the study data, and therefore limiting the effects of nonresponse bias that may have otherwise been present. However, not all nonresponders reply, meaning that there may still be elements of nonresponse bias. Additionally, nonresponders may be contacted at a date many months after the 1-year or 2-year follow-up timelines, and so there may either be missing data for the time points where there is no response, or there may be recall bias where these patients do not give accurate responses on how they were feeling at the time. For example, if a complication arose after when the 1-year follow-up time would have been, but the patient responded after being contacted at a later date, the data may bias the 1-year follow-up data.

Sensitivity Analysis
Sensitivity analyses play roles in the assessment of how robust the conclusions drawn from the primary analysis of data are by examining how the results are affected by changes in methods, models, assumptions, or missing data. When data are missing, the options of sensitivity analysis include either analyzing only complete cases (assume data is missing completely at random) or impute the missing data using imputation methods and redoing the analysis. Most studies should assume that data is missing not at random. There are several single and multiple imputation methods that can be chosen depending on the type of data being collected and depending on whether it is missing at random. 46 Just as it is common in clinical trials, it is of our opinion that all registry studies should incorporate a sensitivity analysis to mitigate the effects of nonresponse bias. However, only 1 study of the 36 included studies performed a sensitivity analysis.
As previously mentioned, the accepted rate of LTFU is disputed for survey studies, ranging from 20% to 40%. Only 1 study 11 achieved a LTFU rate of less than 20%. Several studies quoted LTFU rates of greater than 50%, with 1 study from DKRR quoted a LTFU as high as 82% at 1 year. 26 Although the recommended compliance rates differ between different guidelines, none of them quote a value of LTFU as high as seen in many of the included studies.
A simulation study by Kristman et al. 5 observed that even follow-up rates of 80% (LTFU rate of 20%) were associated with a considerable bias if data were missing not at random. Zelle et al. 13 further performed a simulation study to assess for an acceptable percentage of responders. The study was performed on a polytrauma database from which patient data was deleted randomly at an increasing percentage to simulate various levels of LTFU. It was found that in the 50 simulations performed, only nonresponder rates of 15% or less had no changes in significance on their studied outcome. On the other hand, a nonresponder rate of 20% changed significance in 14 of the 50 simulations they ran on the database.
There are other methods described to alleviate or account for LTFU in arthroscopy studies from the nonregistry studies. One recent example includes the incorporation of machine learning into arthroscopy patient databases where the system can predict the patients that are most likely to not-respond. The study by Kunze et al. 47 looked at 27 different preoperative variables including patient characteristics (such as age, sex, body mass index, race), patient behaviors (such as smoking, alcohol intake, drug use), orthopaedic history (such as previous surgery) and at PROMs to develop 3 models, each of which revealed patients most likely to be LTFU. The 3 models used were "cross-validation," minimizing "Bayes information criteria," and "adaptive selection." Common variables that predicted LTFU among the 3 models were when patients who were of the male sex, nonwhite, smokers, not providing telephone numbers, and having a greater preoperative modified Harris Hip score and international Hip Outcome Tool 12-component questionnaire score (PROMs).
There are also varied conclusions on whether patients LTFU have different outcomes compared to those that are followed-up. An epidemiological study by Ekholm et al. 48 suggested a positive correlation between socioeconomic status and response rates of patients to health interview surveys, although the authors did note there was no significant association between health and nonresponsiveness in their study. Furthermore, a validation study on a local registry was carried out by Lindman et al. 49 reported a larger percentage of replies by nonresponders (76% out of 140 nonresponders) than the validation studies included in this review. Their study found that there was no significant difference in patient-reported hip function according to the PROMs carried out after hip arthroscopy. However, patients who were nonresponders were more likely to be less satisfied with their treatment, younger, and male sex. However, the authors noted that other studies on orthopaedic LTFU analysis did not find significant differences in satisfaction with treatment 50,51 and that studies on nonresponders after arthroplasty procedures reported worse outcomes in terms of knee function in nonresponders. 52,53 A consensus must be reached regarding the handling of missing data and accounting for LTFU of patients included in arthroscopy registries. Because of the compliance rate of arthroscopy registries being lower than those of arthroplasty registries, these issues (despite being of importance) are less pressing for those registries. Although most studies use a dropout analysis or a reference to a validation study, there may be other methods that are more robust in alleviating for nonresponse bias. A follow-up rate of more than 85% may be needed to overcome effects of LTFU and have robust outcome data from registries.
The best form of minimizing nonresponse bias in a registry-based study is to avoid missing data; however, this is not always possible. Possible explanations for poor data collection may include length of questionnaires, method of contacting patients, patients not taking part in follow up due to their outcomes, or patient demographic factors. From the studies included (Table 3), common demographic factors for patients who are LTFU is that they are of male sex and often younger, and therefore more appropriate contact methods (other than questionnaires sent by post or email) may be beneficial. The SNKLR Annual Report from 2019 54 states that new data collection methods such as mobile applications or social media should be explored. The annual report also explores the idea of shortening the length of questionnaires to minimize the time needed to answer each questionnaire. The KOOS questionnaire for the knee for example contains 42 items in 5 sections, whereas the iHOT-12 only contains 12 items as it is a shortened version of the much longer iHOT-33 (a 33 item questionnaire). 55,56 Finally, as arthroscopy is a minimally invasive surgery it has shorter recovery times and fewer complications (such as infection) 57 meaning patients may not feel the need to attend follow-up appointments or answer questionnaires in the years after the procedure.
As well as minimizing missing data, another possible next step after this review could be to perform a consensus study using experts in the field of registries to both determine an acceptable percentage of follow-up compliance and also to determine a universal method for accounting for patient LTFU. Guidelines may be formed from this consensus to guide future arthroscopy registry-based research.

Limitations
There are some limitations to this study. Firstly, only studies from PROM collecting national registries were included in the analysis. Studies from registries that are not nationally centralized may have had other forms of statistical analysis that are used to alleviate for patients who are LTFU. Examples of these include the Multicenter Orthopaedic Outcomes Network and the Kaiser Permanente Anterior Cruciate Ligament Reconstruction Registry, both of which have had numerous studies carried out using data from their patient cohorts. However, studies from such registries were not considered in the search for this review as they do not represent national arthroscopy registries. Second, annual reports for the DKRR and DGOU registries were not available in the English language and had to be translated by Google translate. Native speakers and readers of the languages (Danish and German) were not consulted for the accuracy of the translation. This meant that there was a reliance on the translation software to provide accurate translations. Finally, the search for LTFU analysis was limited to arthroscopy registries, which are still relatively new compared to the arthroplasty registries. However, we are not aware of any consensus on what LTFU analysis method should be used in arthroplasty registries, but this is not as great a concern because the rates of LTFU are lower in arthroplasty registries. 14

Conclusions
Registry studies use inconsistent methods to account for patient LTFU, and rates of patients LTFU are unacceptable.