{'fraction_non_alphanumeric': 0.030792917628945343, 'fraction_numerical': 0.015909674108288425, 'mean_word_length': 5.282375067168189, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Predicting non-response in patient-reported outcome measures: results from the Swiss quality assurance programme in cardiac inpatient rehabilitation

Abstract Background Quality assurance programmes measure and compare certain health outcomes to ensure high-quality care in the health-care sector. The outcome of health-related quality of life is typically measured by patient-reported outcome measures (PROMs). However, certain patient groups are less likely to respond to PROMs than others. This non-response bias can potentially distort results in quality assurance programmes. Objective Our study aims to identify relevant predictors of non-response during assessment using the PROM MacNew Heart Disease questionnaire in cardiac rehabilitation. Methods This is a cross-sectional study based on data from the Swiss external quality assurance programme. All patients aged 18 years or older who underwent inpatient cardiac rehabilitation in 16 Swiss rehabilitation clinics between 2016 and 2019 were included. Patients’ socio-demographic and basic medical data were analysed descriptively by comparing two groups: non-responders and responders. We used a random intercept logistic regression model to estimate the associations of patient characteristics and clinic differences with non-response. Results Of 24 572 patients, there were 33.3% non-responders and 66.7% responders. The mean age was 70 years, and 31.0% were women. The regression model showed that being female was associated with non-response [odds ratio (OR) 1.22; 95% confidence interval (CI) 1.14–1.30], as well as having no supplementary health insurance (OR 1.49; 95% CI 1.39–1.59). Each additional year of age increased the chance of non-response by an OR of 1.02 (95% CI 1.02–1.02). Not being a first language speaker of German, French or Italian increased the chance of non-response by an OR of 6.94 (95% CI 6.03–7.99). Patients admitted directly from acute care had a higher chance of non-response (OR 1.23; 95% CI 1.10–1.38), as well as patients being discharged back into acute care after rehabilitation (OR 3.89; 95% CI 3.00–5.04). Each point on the cumulative illness rating scale total score increased the chance of non-response by an OR of 1.05 (95% CI 1.04–1.05). Certain diagnoses also influenced the chance of non-response. Even after adjustment for known confounders, response rates differed substantially between the 16 clinics. Conclusion We have found significant non-response bias among certain patient groups, as well as across different treatment facilities. Measures to improve response rates among patients with known barriers to participation, as well as among different treatment facilities, need to be considered, particularly when PROMs are being used for comparison of providers in quality assurance programmes or outcome evaluation.


Introduction
In recent years, patient-reported outcome measures (PROMs) have received increased attention as a tool to measure healthcare outcomes. PROMs focus on patients' perspectives and are the only way to address personal satisfaction and healthrelated quality of life (HRQOL). The use of PROMs is associated with many useful applications in clinical care (i.e. shared decision-making and individualized treatment adjustments) and quality improvement [1][2][3]. In external quality assurance programmes, PROMs are used to compare treatment results by different health-care providers; in health-care systems, they are used as indicators for public reporting and performance-based contracting [4,5]. Several PROM initiatives increasingly promote the standardization and institutionalization of the patients' perspective in the health systems (e.g. European Organisation for Research and Treatment of Cancer, International Consortium for Health Outcome Measurement, OECD Patient-Reported Indicator Surveys and Patient-Reported Outcomes Measurement Information System). This focus on the patients' perspective when measuring health was closely related to the implementation of the International Classification of Functioning, Disability and Health (ICF) in 2001 [6] to promote the biopsychosocial model of health. Although PROMs have great potential to anchor the patient perspective more firmly in medical care and quality assurance, their usability has limitations due to high rates of non-response [7,8]. Non-response is not necessarily problematic for the representativeness of data; however, it becomes problematic when non-response is systematic and certain groups are under-represented (non-response bias) [9][10][11]. For national health reporting or comparisons of health providers, it becomes particularly relevant if these subgroups, for example, are in poorer condition or achieve better treatment results than the responders. Systematic non-response biases such as these can lead to over-or under-estimation of health status or treatment outcomes [12,13].
The rehabilitation sector has a long tradition of collecting PROMs due to its early adoption of the biopsychosocial model offered by the ICF. External quality assurance programmes in rehabilitation in Germany and Switzerland have used PROMs as outcome quality indicators for many years already, such as the MacNew Heart Disease questionnaire (MacNew Heart), to measure HRQOL of patients with heart disease [14,15].
In this paper, we analysed MacNew Heart response rates of patients undergoing cardiac rehabilitation in Switzerland. The aim was to explore whether a non-response bias exists and whether this non-response bias is due to specific patient characteristics, features of the clinic or both.

Study design and setting
In 2009, the Swiss National Association for Quality Development in Hospitals and Clinics (ANQ) was founded as a collaboration of all major partners in the Swiss health-care system (Swiss Hospital Association H+, insurance associations, all 26 Swiss cantons and the Principality of Liechtenstein) (www.anq.ch). The ANQ is responsible to conduct external quality assurance for the Swiss inpatient health sector and, amongst others, for rehabilitation clinics. The rehabilitation sector is further divided into several functional departments, for which the ANQ developed detailed measurement schemes. In order to measure HRQOL, the ANQ decided to use PROMs within a number of departments, alongside functional assessments reported by clinicians and performance tests. For quality assurance within cardiac rehabilitation, the chosen PROM is the MacNew Heart.
For this cross-sectional study, we analysed data from all patients within the functional department of cardiac rehabilitation from 2016 to 2019. The cardiac rehabilitation facilities admit patients for inpatient treatment after being in acute care due to cardiac events or surgery, as well as patients with severe chronic cardiac diseases for conservative management. Patients are supported by an interdisciplinary team and receive intensive, multimodal rehabilitation treatment that lasts on average for 3 weeks. After admission and before discharge, the patients were requested to complete the MacNew Heart. The questionnaire was administered as a paper-pencil licensed version in German, French and Italian, which are official languages of Switzerland.
Additionally, clinics submit socio-demographic information, basic medical data (minimum medical dataset of the Swiss federal statistical office [16]) and a measurement for comorbidity based on the cumulative illness rating scale (CIRS) total score [17] for each patient.

Eligibility criteria
All cardiac rehabilitation patients in Switzerland aged 18 years or older, who were admitted into a rehabilitation clinic between 2016 and 2019 and for whom rehabilitation treatment has been completed, were eligible for this study. Cases where treatment was discontinued or shorter than 7 days were categorized as dropouts and not eligible for analysis. Reasons for discontinuation of treatment were death of a patient, transfer to an acute care clinic for more than 24 hours or a request for discontinuation of treatment by the patient.
Furthermore, patients were only included if sociodemographic, basic medical and CIRS data were available. Additionally, the MacNew Heart had to be sufficiently completed by the patient, or refusal or inability of the patient to complete the questionnaire had to be recorded by the clinic personnel (recorded non-response).

Outcome: recorded non-response of the MacNew Heart
The MacNew Heart consists of 27 items; each item can have values between 1 ('severe restriction') and 7 ('no restriction at all'). The total score is calculated from the mean value of all completed items (a minimum of 50% of the items must be filled in).
If a patient is unable to complete the questionnaire, clinics must record this as a 'non-response'. All patients recorded as non-responders at admission or discharge were categorized as non-responders, whereas all patients who filled in the questionnaire at admission and discharge were categorized as responders.

Exposure and covariates
Covariates were collected from socio-demographic and basic medical data, which clinics have to submit to the Swiss federal statistical office for each patient.
We used the following socio-demographic variables as exposure and covariates: age (in years), gender (female or male) and supplementary health insurance (yes or no). Residents in Switzerland may purchase supplementary health insurance on top of their compulsory health insurance, which guarantees a single or double room in hospital and treatment by a head physician. We coded language proficiency of German, French or Italian [L2 (second language) vs L1 (first language) speaker] based on the nationality of each patient. We categorized patients as L1 speakers if their nationality corresponded with a country that has one of these three languages as official language.
Furthermore, we used the following variables to capture comorbidities and health status of the patients: CIRS total score (in points), length of rehabilitation (in days), patient location before admission (home vs acute care), patient location after discharge (home vs acute care) and main diagnosis of each patient (categorized into eight groups of diagnoses). Finally, we adjusted for the year in which the rehabilitation treatment was concluded (2016-2019).
Reporting of our study followed the Strengthening the Reporting of Observational Studies in Epidemiology checklist [18] and the Good Epidemiological Practice guidelines [19].
We used descriptive statistics to determine patients' characteristics in the full sample, as well as stratified by non-responders versus responders. Additionally, to get an impression of how much the response rates vary by each clinic, we showed the percentages of non-response for each clinic in each year.
The associations of patients' socio-demographic and health characteristics with being categorized as non-responder were tested using a random intercept logistic regression model. We estimated odds ratios (ORs) as well as corresponding 95% confidence intervals (CIs) and P values. In the regression model, we used all covariates and stratified by clinic.

Participants
Between 2016 and 2019, data of 29 086 cardiac rehabilitation patients were collected. We excluded 1998 dropouts, as well as 321 patients with incomplete MacNew Heart and 1477 patients with no records for the MacNew Heart. Another 718 patients were excluded because their socio-demographic and basic medical data were missing or incomplete. Our analysis included 24 572 cardiac rehabilitation patients ( Figure 1).

Clinics
All 16 cardiac rehabilitation clinics that treated patients between 2016 and 2019 were included in the analyses. Some clinics submitted data for all 4 years, while others only submitted data for a number of years ( Figure 2).

Descriptive data
Descriptive statistics are shown for the full sample (n = 24 572), as well as stratified by response status [n non-responders = 8172 (33.3%); n responders = 16 400 (66.7%)] in Table 1. Non-responders were more often female, less often L1 speakers of Italian, French and German and had less often supplementary health insurance. Non-responders also scored higher on the comorbidity scale (CIRS total score) and were more likely to be discharged back into acute care.
Non-response rates vary largely between the 16 clinics: some clinics have considerably lower non-response rates while others have non-response rates well above the average ( Figure 2). There is also variation within each clinic over the years.

Inference statistics
The intraclass correlation coefficient (ICC) in the regression model, the marginal and the conditional R 2 showed that about half of the total explained variance (0.10 of 0.196) can be explained by differences at clinic level (Level 2) ( Table 2). The model showed an overall R 2 of 0.196 and thus a high goodness of fit since an R 2 above 0.2 is considered excellent [23].
All socio-demographic predictors were relevant and statistically significant. Female patients had a higher chance of non-response, as well as patients without supplementary health insurance. The chance of non-response was further increased by age. Finally, not being native speaker of German, French or Italian increased the chance of non-response.
A number of medical exposures also showed significant associations: Those admitted directly from acute care had a higher chance of non-response, as well as those being discharged back into acute care instead of being discharged home. Each additional point of CIRS total score increased the chance of non-response. A number of different diagnoses also increased or decreased the chance of non-response, compared to the diagnosis 'Chronic ischaemic heart disease'. Every additional day of rehabilitation treatment increased the chance of non-response slightly. Compared to the year 2016, the chance of non-response remained similar in 2017, decreased in 2018 and increased in 2019.

Statement of principal findings
On individual level, our results show that there were significant differences between patients who completed the MacNew Heart (responders) and those who did not (nonresponders). The most striking finding was that patients who spoke German, French or Italian (language of the questionnaire) as second language had a higher chance of non-response. Women, older patients and patients without supplementary health insurance also had a higher chance of non-response. Being able to afford supplementary health insurance could indicate higher income and could be interpreted as a proxy for higher socio-economic status.
Furthermore, indicators that directly or indirectly measure disease severity were shown to be important. Patients who received longer rehabilitation treatment and those with a higher comorbidity score had an increased chance of nonresponse. Patients admitted to a rehabilitation clinic from an acute care hospital and those discharged back into acute care after rehabilitation were also less likely to respond.
On clinical level, our results show that there were substantial differences in response rates between each facility. In fact, they accounted for about half of the explained variance in the full model.

Strengths and limitations
Several limitations of this study must be considered: The available confounding factors do not cover all potential reasons for non-response such as marital status, education and complete information on socio-economic status. Another limitation is potential misclassification of language status. The only information available was the patient's nationality and not their first language. Patients who are Swiss-born or naturalized Swiss citizens may not have the linguistic resources to answer the questionnaire but, according to this metric, would be considered first language speakers. Moreover, a patient may be fluent in German, French or Italian despite holding a passport from a country where these are not official languages.
Furthermore, a lack of information on clinical conditions (number of beds and staffing levels) means that it was difficult to explain possible differences between clinics. Additionally, the number of Level 2 units (16 clinics) is rather small. Including Level 2 predictors would probably not lead to a better model fit. We thus decided to estimate random intercept logistic regression models despite the small number of Level 2 units since the ICC is rather high [28], and the assumption of independence of errors would thus be violated when using logistic regression models without a second level.
The main strength of our analysis is the dataset on which this study was based. As the Swiss quality assurance programme for inpatient rehabilitation is a comprehensive survey, the data give a thorough overview of all patients in Swiss cardiac inpatient rehabilitation. Socio-demographic and basic medical data were (mostly) available, even for nonresponders. The data furthermore included information about where each patient was treated. We can thus show how much of the explained variance is due to individual-level factors (such as age, gender or language status) and how much variation occurs between the clinics.

Interpretation within the context of the wider literature
Our results are consistent with existing research that found language barriers to be important predictors of non-response when using PROMs. Burrus et al. [10] found being a nonnative speaker led to higher non-response rates in vocational rehabilitation patients in Switzerland. This was also found for patients in an orthopaedic outpatient setting in Australia [24], for a cohort with total joint arthroplasty in the USA [8] and for patients who underwent elective surgery in England [11]. Several national health surveys found higher non-response rates among ethnic minorities, immigrants and their descendants [7,25]. Our finding that patients without supplementary insurance had a higher chance for non-response is in line with a study in which American patients who had Medicare or Medicaid were more often non-responders than patients with commercial medical insurance. Schamber et al. [8] interpreted this as a proxy for lower socio-economic status. Additionally, several studies also identified that more deprived people were less likely to respond [7,11,25,26].
The association of non-response with poorer health status and higher levels of comorbidity was also reported in several surveys. A higher need for personal assistance during completion of the PROM was reported for patients with health-related disabilities [8,10,11,26].
Contrary to our findings, several studies found that nonresponders are more often male [11,25,26], or no association between gender and response rate was reported [7,8]. Our result that elder patients have higher odds for nonresponse is in line with the existing research [7,8,24]; others found the opposite [11,26]. These discrepancies are possibly due to the differing sub-populations that were evaluated.
Based on our results, we cannot conclude whether nonresponders have in fact lower outcomes pertaining to HRQOL after rehabilitation treatment, as information on this was not available for non-responders. Only a few studies could show that non-response was systematically associated with a worse overall health condition [7] or lower results in outcome measurements [27,28].The observed differences in response rates between clinics (the provider effect) are also in line with the existing research. Two studies [12,13] have found large differences between single providers in their analysis of preoperative PROM data from the National Health Service in England. Both studies were unable to report on specific clinic conditions that could potentially be responsible for differing response rates. Further evidence can also be found in an experimental study by Ho et al. [26], where eight cohorts of patients were asked to complete a PROM under differing conditions, resulting in non-response rates between 13% and 81% among the eight groups.

Implications for policy, practice and research
Non-response bias may seem like a predominantly methodological problem. However, it has strong implications for the question of how much confidence we should have in the correctness of our results [9]. High non-response rates in specific clinics lead to small case numbers and could lead to smaller (and non-significant) differences. Additionally, the samples may not represent the true population since certain groups, such as people who are not L1 speakers, less affluent patients and more severely ill patients, are underrepresented.
Results of PROMs and clinic comparisons based on these results must be interpreted carefully, with attention being paid to non-response rates and non-response bias.
Methodologically, these issues can be approached with multiple imputation techniques [13,29]. However, the question is whether imputation on the outcome variable is desirable in a quality assurance context. In fact, complete data should be one aspect of good quality. The indicator would then be the response rate. When action is being taken to increase response rates of PROMs, subgroups with higher chance of non-response should be addressed specifically [7], as non-response bias can remain even with high response rates [9].
The importance of systematic integration of PROMs into organizational procedures and treatment processes should be emphasized in quality assurance programmes. Results should also be discussed with the patient, which can lead to shared decision-making, more adherence and a strengthened selfefficacy of the patients [2,3,30]. This may result in the medical team being more motivated to pay attention to completed questionnaires.
Completing a PROM questionnaire should not burden patients unduly. Offering shorter or proxy versions of the questionnaire for particularly ill patients [31] or questionnaires in different languages for non-native speakers could be an effective strategy for improving response rates for these groups. The easiest way to facilitate this is to offer online questionnaires. They provide instant feedback, and the results can directly be discussed with the patient [32].

Conclusions
We have found significant non-response bias among certain patient groups, as well as across different treatment facilities. Several patient characteristics were associated with non-response, and response rates differed considerably across clinics (provider effect). Measures to improve response rates among patients with known barriers to participation, as well as among different treatment facilities, need to be considered. For quality assurance programmes and outcome evaluations of medical treatments, non-response bias should be a serious concern, particularly when PROMs are being used; their results should be interpreted with caution. However, even though PROMs tend to have high non-response rates, they are necessary for evaluating treatment quality, as several important health outcomes, such as HRQOL, can only be measured from the patient's perspective.

Data availability statement
Data are available upon request from the Head of Rehabilitation at ANQ, Luise Menzi (luise.menzi@anq.ch).

Contributorship
All authors made substantial contributions to data collection, study conception and interpretation of results.