{'fraction_non_alphanumeric': 0.031172584640792735, 'fraction_numerical': 0.011921965317919074, 'mean_word_length': 5.599795640326976, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Psychometric evaluation of instruments measuring the work environment of healthcare professionals in hospitals: a systematic literature review

Abstract Purpose Research shows that the professional healthcare working environment influences the quality of care, safety climate, productivity, and motivation, happiness, and health of staff. The purpose of this systematic literature review was to assess instruments that provide valid, reliable and succinct measures of health care professionals’ work environment (WE) in hospitals. Data sources Embase, Medline Ovid, Web of Science, Cochrane CENTRAL, CINAHL EBSCOhost and Google Scholar were systematically searched from inception through December 2018. Study selection Pre-defined eligibility criteria (written in English, original work-environment instrument for healthcare professionals and not a translation, describing psychometric properties as construct validity and reliability) were used to detect studies describing instruments developed to measure the working environment. Data extraction After screening 6397 titles and abstracts, we included 37 papers. Two reviewers independently assessed the 37 instruments on content and psychometric quality following the COSMIN guideline. Results of data synthesis Our paper analysis revealed a diversity of items measured. The items were mapped into 48 elements on aspects of the healthcare professional’s WE. Quality assessment also revealed a wide range of methodological flaws in all studies. Conclusions We found a large variety of instruments that measure the professional healthcare environment. Analysis uncovered content diversity and diverse methodological flaws in available instruments. Two succinct, interprofessional instruments scored best on psychometrical quality and are promising for the measurement of the working environment in hospitals. However, further psychometric validation and an evaluation of their content is recommended.

instruments. Two succinct, interprofessional instruments scored best on psychometrical quality and are promising for the measurement of the working environment in hospitals. However, further psychometric validation and an evaluation of their content is recommended.
Key words: work environment, organizational culture, hospital, instruments, psychometric properties, systematic review Purpose A positive work environment (WE) for healthcare professionals is an important variable in achieving good patient care [1] and is strongly associated with good clinical patient outcomes, e.g. low occurrence of patient falls and pressure ulcers, good pain management, low hospital mortality and hospital acquired infections rates [2,3]. Associated with efficiency, e.g. fewer re-admissions and adverse events [2][3][4], a positive WE is a prerequisite for a safety climate and a high performing organization that finds quality improvement a part of daily practice [2,5,6]. Research shows that when healthcare professionals perceive a positive WE, they have more job satisfaction and are therefore likely to stay longer; fewer staff will suffer burnout or workrelated stress [7][8][9][10].
In general, WE is defined as the inner setting of the organization FOR which staff work [11]. In healthcare, a positive WE is defined as a setting that supports excellence and decent practices that strive to ensure health, safety and the personal well-being of staff, support quality patient care and improve the motivation, productivity and performance of individuals and organizations [12]. Pearson et al. [13] explains the relevant elements of WE as 'a workplace environment characterized by: the promotion of physical and mental health as evidenced by observable positive health and well-being, job and role satisfaction, desirable recruitment and retention rates, low absenteeism, illness and injury rates, low turnover, low involuntary overtime rates, positive inter-staff relationships, low unresolved grievance rates, opportunities for professional development, low burnout and job strain, participation in decision-making, autonomous practice and control over practice and work role, evidence of strong clinical leadership, demonstrated competency and positive perceptions of the work environment including perceptions of work-life balance. ' A positive WE stems from respect and trust between colleagues at all levels, effective collaboration and communication between all educational levels within a profession, different disciplines and working on different departments [14], recognition for good work, a safe atmosphere, positive climate and support from management [5,6].
Measuring WE is not easy, since this multidimensional concept encompasses diverse elements [13,15]. Some WE measuring instruments focus on specific professions (e.g. nursing [16][17][18], physicians [19,20], residents [21], management [22,23]) or specific wards (e.g. intensive care units [17], critical care [24], cardiac care [25]) or include only one or two aspects of WE (e.g. ethics, social climate [26], organizational culture [27][28][29], organizational climate [30]). Achieving a positive WE is not just up to the members of one profession in one department, but a challenge for a team with members from various professions, roles or departments and even organizational boundaries [14,31]. WE is not the sole responsibility of management, but of management and healthcare professionals together. Therefore, a WE measurement instrument should measure all members of a team and not just those from one profession, one department or one or two aspects of WE.
If hospitals pursue systematic and objective insights into their WE with a valid, reliable and succinct measurement tool, they would gain an understanding of the influential factors that would allow them to improve WE for the benefit of their patients, staff and organization. The aim of this systematic review was to assess instruments that provide valid, reliable and succinct measures of health care professionals' WE in hospitals.

Data sources and study selection
To find an instrument that staff can use to assess their WE, we performed a three-step study. First, we systematically searched the literature to detect all available WE measuring instruments. Second, we assessed the content of these instruments. Third, we assessed instrument quality with the COSMIN guidelines [32,33], particularly their psychometric properties. To ensure optimal clarity and transparency, we used the PRISMA reporting guideline to structure this paper [34].
Step 1: systematic literature searches One researcher (SM) and a librarian systematically searched Embase, Medline Ovid, Web of Science, Cochrane, CENTRAL, CINAHL, EBSCOhost and Google Scholar using key words and their synonyms: 'WE,' 'organizational culture' and 'measurement' (see Supplementary File 1) from inception through December 2018. No search limits were used for language, publication date or type of research. Titles and abstracts of retrieved papers were independently reviewed for inclusion by two researchers (SM and CO). The inclusion criteria were: (i) written in English, the paper describes the development of an original WE measuring instrument for healthcare professionals in hospitals; (ii) the instrument is not a translation of another instrument; (iii) the paper describes psychometric properties with at least some form of construct validity and reliability. Given our focus on the WE of all hospital staff, we excluded papers describing WE instruments in a single profession in one department.
The reviewers discussed to the point of consensus any differences in their assessments of potentially eligible papers. Full versions of eligible papers were then scrutinized independently by three researchers (SM, CO and GB) and cited references were assessed to find additional instruments. Disagreements on these assessments were discussed with a fourth researcher (AMW) until consensus was reached.

Data extraction
Step 2: content assessment Based on a pre-defined data extraction form two researchers independently (SM, CO, GB or AMW) extracted the study context and instrument content. Study context included research design, country, clinical setting, number and types of health care staff. Instrument content included primary goal, measurement type, focus of interest and number of items, subscales, sample, and study setting.
Next, to enable comparative analysis of the contents, two researchers (SM and CO) independently sorted and clustered all items/subscales of the instruments into elements. Their content analyses were discussed up to consensus by the whole research team.
Hospital work environment • Review Article Table 1 Definitions of measurement properties [33] Measurement property Definition

Content development
The degree to which the content of a measurement instrument is an adequate reflection of the construct to be measured Internal consistency The degree to which different items of a (sub)scale correlate and measure the same construct (interrelatedness) Reliability The extent to which scores for persons who have not changed are the same for repeated measurement under several conditions Structural validity The degree to which the scores of an instrument are an adequate reflection of the dimensionality of the construct to be measured Criterion validity The degree to which the scores of an instrument are an adequate reflection of a 'golden standard' Hypothesis testing for construct validity The degree to which the scores of the instrument are consistent with the hypotheses based on the assumption that the instrument measures the construct to be measured Measurement error The systematic and random error of a patient's score that is not attributed to true changes in the construct to be measured Responsiveness The ability of an instrument to detect change over time in the construct to be measured Step 3: quality assessment To appraise the methodological quality of the instruments, we assessed their psychometric properties: measurement development, internal consistency, reliability, structural validity, criterion validity, hypothesis testing for construct validity, measurement error and responsiveness. We used the consensus-based standards for the selection of health measurement instruments (COSMIN) risk of bias checklist [32,35]. The COSMIN checklist was developed to assess the methodological quality of single studies included in systematic reviews of patient-reported outcome measures (PROM) [32]. Although the subject of our review is a staff outcome measurement and not a PROM, this assessment method is useful because the purpose remains the same: screening for the risk of bias. The COSMIN risk of bias checklist is a modular tool, which means that only the measurement properties that were described in a paper were assessed [32]. COSMIN contains two boxes on content validity. The second box focuses on detailed content validity development issues and is not suitable for the type of studies included in this review. Therefore, we used only the first box, 'PROM development.' Table 1 lists the definitions of properties as applied in this review. Two researchers (SM and CO) appraised quality on a fourpoint scale (very good-adequate-doubtful-inadequate). Their ratings were independently crosschecked by two other researchers (GB and AMW). The methodological quality score for each psychometric property was determined by the lowest rating of any item in that category [32]. When applicable, the measurement properties were rated by the 'criteria for good measurement properties' as described by Mokkink et al. [32]. Properties were judged as 'sufficient' (+), 'insufficient' (−) or 'indeterminate' according to the COSMIN standards [35].

Results of data synthesis
The search strategy (see Figure 1) yielded 6397 individual papers. After screening the titles and abstracts, 6305 papers were excluded because they did not describe the development of an original instrument to measure the WE of healthcare professionals or did not provide psychometric details. This resulted in 92 potentially relevant papers eligible for full text screening. After full text screening, another 57 papers were excluded based on the inclusion criteria. Assessing the references cited in the included papers found one other relevant study for a total of 37 included papers.
Other measurement properties were too scattered for both method and method quality to assess criterion validity or hypothesis testing. Other fundamental measurement properties were performed arbitrarily and if available, the quality can be considered as doubtful. Best overall quality assessment was found for the culture of care barometer (CoCB) [65] and the Picker Employee Questionnaire for Midwives [72] because of their overall adequate score on COSMIN criteria and sufficient statistical outcome for internal consistency. That said, measurement properties such as reliability, hypothesis testing and criterion validity have not yet been established for these relatively new instruments (Table 4).

Discussion
The aim of this review was to assess WE instruments and learn which ones provide valid, reliable and succinct measures of health care professionals' WE in hospitals. We identified 37 studies that report  on the development and psychometric evaluation of an instrument measuring healthcare professionals' experience of WE in hospitals. The number of instruments found, even using tight inclusion criteria, reflects the importance of the WE concept in the past 35 years.
Despite new management structures, the greater focus on cost containment, and the change in focus from profession-centeredness to patient-centeredness have not influenced the importance of WE measurement [6,73]. Especially rising attention for patient safety and high-performing organizations steered the importance of WE measurement. However, over the years, WE measurements have been made under different names, different elements and focus. Although elements did overlap, we could not identify one clear set to measure WE. Therefore, it is not possible to conclude which elements contribute more to the WE construct based on the assessment of the instruments. Additionally, most studies used a sample from the nursing domain, especially nurses [37, 38, 40, 46-51, 53, 55, 71], whereas a positive WE is team-based and teams in hospitals contain more than one profession, different educational levels and specialisms [14]. We found methodological flaws in most of the papers reporting the development of WE instruments. The most relevant shortcomings are the lack of information on scale development, failing to fully determine structural validity by confirmative factor analysis and failing to establish such psychometric properties as 'reliability,' 'criterion validity,' 'hypothesis testing,' 'measurement error' and 'responsiveness.' This made drawing firm conclusions on the validity and reliability of the 37 instruments included in this review hardly possible. Just five instruments scored 'adequate' or 'very good' on the COSMIN risk of bias checklist on all of the applied properties [42,50,54,65,72]. Of the five, only the short questionnaire for workplace analysis (KFZA) [54] and the CoCB [65] are both generally applicable and succinct, with an item total below the mean of this review. Both instruments are recent developments, which could suggest that scientists are paying more attention to the (reporting of) methodology of measurement instrument development.

Limitations
Some limitations of this study warrant consideration. First, to compare instrument content, the item and subscale descriptions of the individual instruments were mapped into 48 elements. Some details of instruments may possibly have been lost in the mapping process. Second, we sought original development and validation studies for this review, which may mean that other publications that discuss other psychometric properties of the included instruments were left out. Third, we searched for instruments intended to measure the WE in hospitals. Nevertheless, a large group of studies used samples from predominantly one discipline (e.g. nurses or nursing assistants [37, 38, 41-45, 51-53, 71]), and some instruments were developed specifically for one discipline (e.g. nursing [39,40,46,[48][49][50]55]). Given that nurses are the largest professional group in hospitals, our search had to include measurement instruments for nursing. However, our assessment focused on instruments measuring WE in general and thus excluded instruments measuring a specific type of nursing or department.

Implications for research
To address methodological issues in the development process of instruments, it is important that instruments provide an understanding of the construct to be measured. Therefore, it is crucial that healthcare professionals participate actively in next phase. Clear definitions of items and categories would be helpful in creating distinct construct definitions and thus obtain a better understanding of what should be included in a WE measurement instrument to provide relevant, comprehensible and meaningful information [74,75]. Some instruments found in this review already perform well, so we do not recommend developing new instruments. Rather, we advise scrutinizing the methodology of existing instruments using the COSMIN guidelines. For instance, we suggest performing confirmative factor analyses to check whether the data fit the proposed theoretical model for WE, and to determine the responsiveness of WE instruments in longitudinal research [32,35].

Implications for practice
A positive healthcare WE is vital for high-performing healthcare organizations to provide good quality of care and retain a happy, healthy professional workforce [2,6,7,76] so obtaining periodical insight into WE assessment on the team level is important [72]. Preferably, the WE instrument should facilitate teams and management to improve the WE, e.g. by deploying, monitoring and evaluating focused interventions. Besides taking valid, reliable measurements, the instrument should provide clearly relevant information for healthcare professionals [6,77,78]. Research shows that if an instrument provides information for use as a dialog tool, teams will become actively engaged in improving their WE [65]. Especially, the CoCB [65] is designed to do this.
Based on the assumption that instruments containing more than one construct measured with the same method are at risk of overrated validity [74,75], the outcomes of the CoCB should always be used in combination with other managerial information, e.g. patient quality data or data on personnel sick leave and job satisfaction.

Conclusion
The findings of this systematic review have potential value in guiding researchers, healthcare managers and human resource professionals to select an appropriate and psychometrically robust instrument to measure WE. We have demonstrated content diversity and methodological problems in most of the currently available instruments, highlighting opportunities for future research. Based on our findings, we draw the cautious conclusion that more recently developed instruments, such as the CoCB [65], seem to fit current reporting demands for healthcare teams. However, we suggest investing in improving their psychometrical quality.