{'fraction_non_alphanumeric': 0.029685206328494536, 'fraction_numerical': 0.01984450606208884, 'mean_word_length': 5.013076168682576, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Effect of e-Learning and Repeated Performance Feedback on Spirometry Test Quality in Family Practice: A Cluster Trial

PURPOSE Spirometry has become an indispensable tool in primary care to exclude, diagnose, and monitor chronic respiratory conditions, but the quality of spirometry tests in family practices is a reason for concern. Aim of this study was to investigate whether a combination of e-learning and bimonthly performance feedback would improve spirometry test quality in family practices in the course of 1 year. METHODS Our study was a cluster trial with 19 family practices allocated to intervention or control conditions through minimization. Intervention consisted of e-learning and bimonthly feedback reports to practice nurses. Control practices received only the joint baseline workshop. Spirometry quality was assessed by independent lung function technicians. Two outcomes were defined, with the difference between rates of tests with 2 acceptable and repeatable blows being the primary outcome and the difference between rates of tests with 2 acceptable blows being the secondary outcome. We used multilevel logistic regression analysis to calculate odds ratios (ORs) for an adequate test in intervention group practices. RESULTS We analyzed 1,135 tests. Rate of adequate tests was 33% in intervention and 30% in control group practices (OR = 1.3; P=.605). Adequacy of tests did not differ between groups but tended to increase with time: OR = 2.2 (P = .057) after 3 and OR = 2.0 (P = .086) in intervention group practices after 4 feedback reports. When ignoring test repeatability, these differences between the groups were slightly more pronounced: OR = 2.4 (P = .033) after 3 and OR=2.2 (P = .051) after 4 feedback reports. CONCLUSIONS In the course of 1 year, we observed a small and late effect of e-learning and repeated feedback on the quality of spirometry as performed by family practice nurses. This intervention does not seem to compensate the lack of rigorous training and experience in performing spirometry tests in most practices.


INTRODUCTION
S pirometry has become an indispensable tool for primary care professionals to diagnose and monitor chronic respiratory conditions. 1,2 Recent studies indicate that, when implemented in primary care, spirometry is a valid test 3 that leads to increased rates of respiratory diagnoses and may improve disease management in the United States and elsewhere. [4][5][6][7] Although spirometry can be made available in several ways, from a practical point of view having good-quality tests performed in the practice itself is the preferred mode. 8 In the Netherlands approximately 62% of family practices own a spirometer (the remaining practices having access to spirometry facilities elsewhere), and the rate of spirometry tests performed in family practices has tripled in the past couple of years. 9 Recent surveys in the United States indicate that 47% to 75% of family physicians use spirometry. 10,11 Lack of spirometry training appears to be an important barrier to the use of spirometry in US family practices. [10][11][12] Performing good quality spirometry requires proper training and well-standardized procedures, 13 which may be diffi cult to achieve in a family practice. 14 Using different sets of criteria, investigators have studied the quality of spirometry tests in family practices and reported rates of adequate tests ranging from only 3% to up to 80%. 5,[14][15][16][17] Although there are some indications that training and performance feedback may increase the quality of spirometry in nonlaboratory settings 18,19 , only 1 controlled intervention trial has been published from a real-life family practice setting. 15 In the study reported in this article, we investigated whether a combined intervention of e-learning and subsequent bimonthly performance feedback after a baseline spirometry workshop would improve test quality in family practices in the course of 1 year.

Study Design
The study was a cluster controlled trial, with family practices as clusters for patients in whom 1 or more spirometry tests were performed during a 1-year observation period (Clinicaltrials.gov Protocol Registration System: NCT00962455; http://www.clinicaltrials.gov). All practices involved were nonacademic practices that have a working agreement with a local hospital (Elkerliek Hospital, Helmond, The Netherlands) regarding support in spirometry training, test execution, and interpretation. All practices have a PC-based spirometer (SpiroPerfect, Welch Allyn, Delft, The Netherlands). The spirometry tests are electronically submitted by the practices and can be accessed by the hospital's lung function technicians and chest physicians. The Medical Ethics Committee of the Elkerliek Hospital approved the study (fi le number 07-393).

Recruitment, Sample Size, and Group Allocation
Practices were recruited through a mailing to all family practices that collaborate with the Elkerliek Hospital. Cluster sample size calculation 20 showed that 19 practices were needed to demonstrate a 25% difference in the rate of adequate spirometry tests. Assumptions for the power calculation were as follows [14][15][16] : 30% adequate spirometry tests in control practices; 12-month study duration; average of 2 spirometry tests per practice per week; intracluster correlation coeffi cient (ICC) of 0.15; α level of .05; 1 -β of 0.80. Although we anticipated a learning curve in intervention effects over time, we did not take this into account in the sample size calculation. After recruitment, practices received a questionnaire to inquire about their spirometry details and were allocated to either the intervention or usual practice group through computerized minimization, a method of ensuring balance between groups for several prognostic factors, even in small samples. 21 Allocation was stratifi ed by the average weekly number of spirometry tests in the past year and by practice nurses' years of spirometry experience.

Intervention and Control Conditions
All practice nurses involved in the study attended a 2.5-hour baseline workshop to refresh their spirometry knowledge and skills. Current criteria for adequate spirometry tests 13 were discussed, and execution of tests was practiced. At the end of the workshop, all nurses were encouraged to continue or increase their usual frequency of spirometry testing. Intervention practice nurses stayed for an additional 30 minutes to (1) receive a copy of the e-learning CD-ROM Spirometry Fundamentals (University of Washington, Seattle, Washington), (2) be instructed in how to study the CD-ROM in the next 4 weeks, and (3) discuss details of the upcoming bimonthly feedback reports-including the way these reports should be interpreted and discussed during telephone calls with the lung function technician who had been assigned as their coach. The feedback reports focused on the spirometry tests performed in the past 2-to 2.5-month period and, after the fi rst period, on comparison with previous periods. All practices were instructed that (1) only trained nurses should perform spirometry, (2) all tests performed as a part of routine patient care should be submitted, and (3) each submitted test should consist of the 3 best blows.
The observation period for the trial started immediately after the spirometry workshop (on November 6, 2007) and lasted exactly 12 months. A new feedback period started on the day the feedback reports that were based on the tests from the previous period had been e-mailed to the intervention practices. After electronic submission, the quality of prebronchodilator tests from intervention practices was assessed by one of the Elkerliek Hospital's lung function technicians, using a checklist based on recent recommendations. 13 Because the lung function technicians could invest a limited number of hours to assess spirometry tests for the study, 25 tests per practice per period was set as the maximum number of tests on which a feedback report could be based.

Spirometry Quality Assessment and Study Outcomes
Three experienced lung function technicians who were not otherwise involved in the study assessed de-identifi ed printouts of all submitted tests. These technicians were instructed on how to score spirometry test quality, 13 practiced on 10 tests, and discussed their scoring and experiences. Two technicians volunteered to assess all tests for the trial. Interobserver agreement between these 2 technicians was assessed in a random sample of 80 tests 22 and was deemed suffi cient (Cohen's κ = 0.79). In case of disagreement, the third technician assessed the test. Tests were presented to the technicians in random order, and they were blinded to each other's assessments and to information on the practice or study period to which a test belonged.
We only used prebronchodilator tests from patients older than 10 years. The primary outcome for the study was the proportion of tests with 2 or more acceptable blows that were also repeatable for both FEV 1 (forced expiratory volume in 1 second) and FVC (forced vital capacity). 13,15 An adequate forced blow has a good start, satisfactory duration of exhalation, and is free from artifacts (eg, cough, glottis closure, obstruction of mouthpiece). Because the feedback report focused strongly on acceptability of blows and less so on repeatability between blows, we studied differences in the proportions of tests with 2 or more acceptable blows as a secondary outcome.

Statistical Analysis
Multilevel logistic regression analysis with family practice as cluster level and a compound symmetry matrix correlation structure was used to test differences in the odds of an adequate spirometry test (GLIMMIX procedure, SAS 9.2, SAS Institute, Cary, North Carolina). Odds ratios (ORs) derived from the logistic regression models express the odds (and 95% confidence interval [CI]) of an adequate test in intervention relative to control practices. The logistic model also included the period in which the test had been performed and covariates related to test quality (ie, age, sex, and severity of airfl ow obstruction). 16 ICCs were calculated for the primary and secondary outcomes. Statistical tests were 2-sided; P <.05 was considered statistically signifi cant. Figure 1 displays the recruitment and allocation of practices and reasons for excluding spirometry tests from the analysis. One of the 9 intervention practices dropped out before the fi rst feedback report was provided. A total of 490 prebronchodilator spirometry tests from intervention and 645 tests from usual prac- a In their mutual working agreement, the family practices and the Elkerliek Hospital agreed that patients younger than 16 years should be referred to the hospital when a spirometry test is required. Even so, there were 27 tests from patients in this age-group in our dataset. Because obtaining a good-quality spirometry test in young children requires specifi c training and skills from nurses, we excluded tests from children younger than 10 years from the analysis. b Because the lung function technicians from the Elkerliek Hospital could invest a limited number of hours to assess spirometry tests for the study, 25 tests per practice per period was set as the maximum number on which a feedback report could be based. c All tests were color-printed in the pulmonary function laboratory of the Elkerliek Hospital and de-identifi ed by the investigators before the blinded, randomized outcome assessment by the independent lung function technicians. A small number of tests were erroneously printed twice. d Tests that had been submitted by family practices in which the independent lung function technicians recognized technical malfunction of the spirometer or spirometry software. These tests were excluded from further analyses because the lung function technicians could not judge the nurse's performance.

Effects on Primary and Secondary Outcomes
The coaches' records showed that all intervention practice nurses reported to have studied the spirometry CD-ROM shortly after the baseline workshop, and that instructions in the feedback reports to repeat particular CD-ROM modules were followed in 92% of cases. Table 2 shows the rate of test adequacy for the respective   quality criteria. Overall, poor start (21.6%) and insuffi cient duration of blows (38.8%) were the predominant reasons for test inadequacy, but no statistically signifi cant differences between intervention and control group practices were observed. For the primary outcome, the rate of adequate tests in the fi rst 2 months (ie, after the baseline workshop and initial e-learning in intervention practices, but before the fi rst feedback report) was 30.4% in intervention and 25.0% in control practices. The odds ratio for an adequate test in intervention relative to control practices was 1.3 (95% CI, 0.5-3.2; P = .605). For all 5 study periods combined, the rate of adequate tests was 32.9% in the intervention group and 29.8% in the control group (OR = 1.2; 95% CI, 0.6-2.5; P = .663). Figure 2 (panel A) shows that the adequacy of tests gradually increased over the consecutive study periods in the intervention group relative to the control group. In the fi fth and fi nal study period, rates of adequate tests for the primary outcome were 43.3% and 34.1% for intervention and control practices, respectively. The odds ratio of an adequate spirometry test in the intervention group was not statistically signifi cant during the whole test period, but approached the threshold for signifi cance after 3 feedback reports: OR = 2.2 (95% CI, 1.0-5.0; P = .057) in period 4 and OR = 2.0 (95% CI, 0.9-4.5; P = .086) in period 5. When repeatability between blows was ignored in the defi nition of test adequacy, the differences between the study groups were more pronounced: OR = 2.4 (95% CI, 1.1-5.3; P = .033) in period 4 and OR = 2.2 (95% CI, 1.0-4.8; P = .051) in period 5 (Figure 2, panel B). Supplemental Figure  1 (available at http://www. annfammed.org/cgi/content/ full/9/4/330/DC1) displays the rate of adequate spirometry tests per practice and per period for the primary outcome.

Summary of Main Findings
In a cluster controlled trial we investigated whether a combined intervention of e-learning and bimonthly performance feedback was able to improve the quality of spirometry tests in Dutch family practices that had already implemented spirometry within their regular patient care. Although we did not observe an intervention effect for the overall 12-month observation period, a modest increase in the rate of adequate tests emerged after 3 feedback reports. This increase led to a trend toward an approximately 10% higher rate of adequate tests in the fi nal 4 months of the observation period in the intervention group. As in previous studies, 15,16 not meeting end-of-test criteria was the predominant reason for test inadequacy.

Comparison With Existing Literature
Certifi ed and skilful lung function technicians can be expected to produce adequate spirometry tests in 80% or more of adult patients tested. 23 Although our data show that some family practice nurses may be able to achieve rates of adequate tests in 50% to 75% of tests (Supplemental Figure 1), it is unlikely that most practices will be able to achieve and maintain such a performance level. A recent Australian study showed that delegating spirometry to well-trained and experienced visiting nurses substantially improved spirometry adequacy. 14 Adopting this (or a similar) model may overcome practice nurses' lack of training, experience, and routine-which are essential factors for good-quality spirometry. 24 Studies conducted in specialized 19,25 as well as in primary care 15,18 settings suggest that performance feedback is able to improve spirometry test quality. Our fi ndings support such an approach, although expectations should be fairly modest, and it may take a rather long breath and more intensive (certifi ed) training programs to achieve satisfactory results.
In our view, obtaining suffi ciently reliable and clinically useful spirometry tests-not necessarily perfect tests-is what family practices should be striving for. The impact is of inadequate spirometry tests on patients' diagnoses and management in family practice is currently unclear.

Strengths and Limitations
A particular strength of our study was that it was undertaken in a real-life setting in which a group of family practices had implemented spirometry for several years. Compared with national fi gures, our group of practices was representative for the Netherlands in terms of spirometry experience, 9 but smaller (ie, singlehanded and duo) practices were underrepresented. 26 All practices involved in the evaluation participated in the working agreement with the local hospital, which may limit the generalizability of our fi ndings. Other strengths are the high participation rate, the individualized feedback, the rigorous method of outcome assessment using multiple blinded experts, the clustered trial design and multilevel analyses, and the stratifi cation by spirometry experience of practice nurses. A limitation was the 12-month duration of the study. Although 12 months would seem like a suffi cient observation period, the intervention appeared to start having effect after 3 feedback reports. A few additional months of follow-up might have shown either a continuing upward trend, a plateau, or a regression of intervention effects.
During the course of 1 year we found a rather small and late effect of baseline e-learning and repeated performance feedback on the quality of spirometry tests by practice nurses who perform spirometry as a part of regular patient care in family practices. This intervention does not seem to be able to fully compensate for their lack of rigorous training and experience in performing spirometry tests. Other models to provide family practices with good-quality spirometry should be explored.
Funding support: This study was funded by QUARTZ, a regional, integrated care support service; the Elkerliek Hospital; and the Department of Primary and Community Care of the Radboud University Nijmegen Medical Centre, the Netherlands.
An oral presentation at the International Primary Care Respiratory Group (IPCRG) Conference in Toronto, Canada, 2-5 June 2010.