{'fraction_non_alphanumeric': 0.03827540695116586, 'fraction_numerical': 0.029256489221293445, 'mean_word_length': 4.921073196144829, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Evaluating the impact of coronavirus disease on burnout among healthcare workers using maslach burnout inventory tool: A systematic review

Background Burnout has been prevalent among healthcare workers (HCWs). However, the effect of the Coronavirus Disease (COVID-19) pandemic on this phenomenon in HCWs is unclear. Objective This systematic review aims to evaluate the impact of COVID-19 on burnout of HCWs using Maslach Burnout Inventory (MBI). Methods A systematic search was performed on PubMed database for articles published between 1 December 2019 and 30 June 2021. Search strategy combined terms for HCWs, COVID-19, burnout, and MBI. The main outcome of interest was burnout, including both mean prevalence and MBI scores for high emotional exhaustion (EE), high depersonalisation (DP) and low personal accomplishment (PA). Results Four cohort studies, 90 cross-sectional studies and one randomised-controlled trial were included for review. Only one cohort study compared burnout data among HCWs before and during COVID-19. It reported a statistically significant increase in mean EE and PA scores from 21.9 to 24.8 (p = .001), and 42.7 to 48.7 (p = .001), respectively. The remaining studies only evaluated burnout data during COVID-19 but were missing burnout data prior to the pandemic for comparison. Across these studies, the overall mean prevalence of burnout among HCWs was 39.95%, with mean MBI EE scores of 22.07, DP scores of 7.83, and PA scores of 32.53. Burnout outcomes were generally comparable across specific healthcare professions such as doctors and nurses. Conclusion Whilst quality research elucidating the effect of pandemic on burnout is lacking, current burnout prevalence among HCWs during COVID-19 is notable.

evaluates oneself negatively and feels discontented about one's work or performance. 2,4 Several studies conducted between 2005 and 2017 reported that healthcare workers (HCWs) are at high risk of burnout, 6 with the prevalence of burnout ranging from 30% to 70% among doctors, 7,8 33.3% among nurses, 9 and often more than 50% among pharmacists. [10][11][12] This is likely due to the immense emotional and psychological burden resulting from HCWs' responsibility for their patients' overall health. Burnout symptoms may manifest in HCWs in terms of negative mental health effects such as stress, psychological distress, 13,14 and anxiety, 5,15 or job withdrawal behaviors such as absenteeism and reduced productivity. 5 Since December 2019, the global Coronavirus disease (COVID-19) pandemic has strained healthcare systems and HCWs. [16][17][18][19] Healthcare institutions face occupational stressors such as having risks of insufficient personal protective equipment, inadequate manpower and increased workload. 17,18,20 Furthermore, HCWs face increased emotional demands such as worry for their own health [21][22][23] or fear of infecting others, 21,23 and these worries can contribute to burnout. 24 Despite increased awareness of HCWs' well-being during the pandemic, 25 the impact of COVID-19 on burnout of HCWs remains unclear. This systematic review aims to evaluate the impact of COVID-19 on burnout of HCW measured using Maslach Burnout Inventory (MBI), a validated tool. 4,26,27 Methods A team of four researchers performed a systematic review of primary literature evaluating the impact of COVID-19 on the burnout of HCWs. This review was conducted in accordance with "Preferred Reporting Items for Systematic Reviews and Meta-Analyses" (PRISMA) guidelines. 28

Eligibility criteria
We retrieved articles that contained at least one of the following terms: (a) HCWs, (b) COVID-19 and (c) burnout or MBI scores. Our target population was HCWs. The exposure and/or comparator was presence of COVID-19, while the primary outcome of interest was MBI score. Studies were included if they recruited HCWs and had burnout data measured using MBI during COVID-19. Studies that were missing pre-COVID 19 historical comparator data were also included for the descriptive analysis. Studies that were not written in English language were excluded. Protocols, commentaries, editorials or unpublished studies were also excluded.
Search strategy, information sources and study selection On 19 July 2021, one researcher performed a comprehensive systematic search on PubMed electronic database using the following keywords: "health personnel" OR "health occupations" AND "COVID-19" OR "SARS-CoV-2" AND "burnout" OR "Maslach Burnout Inventory" OR "MBI" OR "mental health". The full list of search terms can be viewed in Supplementary Appendix Table A1.
Studies published between 1 December 2019 to 30 June 2021 were retrieved. Additional studies from the reference lists of included reviews were also retrieved. Screening of titles and abstracts of studies and removing of duplicates were performed manually by two independent reviewers. Any disagreements throughout the process were resolved through consensus with a senior researcher. Subsequently, the same reviewers performed full-text screening of the remaining studies to assess their eligibility for inclusion.
Data collection process, data items and quality appraisal Two independent reviewers extracted data from included studies and compiled them in a standardized Excel template. The data extracted included: name of author(s), publication year, country, study design, period of data collection, sample size, age (mean ± standard deviation), sex distribution, healthcare profession(s) involved and burnout outcomes. A sample list of the data collected from included studies can be viewed in Supplementary Appendix Table A2.
Risk of bias (ROB) was performed simultaneously during full-text screening and data extraction of included articles. The quality of studies was assessed by the same two reviewers using the 8-item, 11-item, and 13-item Joanna Briggs Institute (JBI) critical appraisal tools for cross-sectional studies, cohort studies and randomized controlled trials, respectively. 29 While the ROB was assessed for all studies, measuring the extent of exposure to COVID-19 stressors could not be done reliably and objectively. Hence, a score of zero was allocated for these items. Studies were categorised as low (13)(14)(15)(16), moderate (7)(8)(9)(10)(11)(12) and high ROB (0-6) based on their tabulated scores. 30 The ROB assessment for all included studies can be viewed in Supplementary Appendix Table A3.

Study outcomes and summary measures
Maslach Burnout Inventory is a validated tool to measure burnout, comprising a 22-item, self-reported questionnaire based on a 7-point Likert scale that assesses EE, DP, and PA subscales. 4,26 There are several variations of the MBI, of which MBI-Human Services Survey (MBI-HSS) and MBI-Human Services Survey (Medical Personnel) (MBI-HSS(MP)) are validated and most used for HCWs. 31,32 In our review, studies that used MBI-HSS or MBI-HSS(MP) were grouped together under MBI-HSS due to their near identical question set, differing only in choice of noun describing clients as patients. 32 Another included variation was the MBI-General Survey (MBI-GS).
To our knowledge, based on previous systematic reviews published on burnout evaluated with MBI, there is often no standardized definition of burnout across different studies. [33][34][35][36] There is also a lack of consensus on the MBI cutoff scores defining high EE, high DP, and low PA, with MBI's authors acknowledging the lack of diagnostic validity of cutoff scores in defining burnout. 37 Despite these issues limiting comparison across different studies, we sought to organize burnout data from identified studies to quantify the mean prevalence of burnout and mean MBI scores for its three subscales (EE, DP and PA) as defined by the study's authors. For each identified study, we obtained the authors' defined MBI cut-off scores for high EE, high DP, and low PA, and used these cut-off scores to review the said study's data. Corresponding authors of studies that did not report cut-off scores or had missing burnout data were contacted by email for further clarification. These studies were omitted if the authors did not respond. As burnout among HCWs is more commonly defined as high EE score of ≥27, high DP ≥ 10, and/or low PA ≤ 33 based on MBI, studies that used these specific MBI cut-off scores for burnout were planned to be grouped together to assess the pooled effects and for subsequent sensitivity analysis.
Some studies reported median values for burnout prevalence and MBI scores. Based on the Central Limit Theorem, 38 if the sample size of the individual study exceeded 30, we estimated the median value to be close to the mean, hence these median values would be pooled with mean estimates listed in other studies.

Data analysis and synthesis of results
Burnout prevalence and MBI scores for EE, DP and PA were evaluated as continuous data, with the data presented as weighted mean (minimum -maximum). For meta-analysis, we planned to assess the heterogeneity of included studies via visual inspection of forest plots, calculated I 2 statistic and 95% confidence interval. Similar studies would be grouped together for sub-group analyses based on study design and healthcare profession. Pooled means of burnout prevalence and MBI scores would be calculated using fixed-effects and random-effects models, for similar studies and those with large heterogeneity, respectively.

Publication bias and additional analyses
Assessment of publication bias across studies using funnel plot and additional analyses for profession were conducted.

Study selection
A total of 6150 articles were retrieved for title and abstract screening, of which 2337 were retained for full-text review. Total of 95 articles were eventually included in this review. Figure 1 shows the PRISMA Flow Diagram for the inclusion and exclusion of articles.

Study characteristics
Study design and demographics. A total of 95 studies, comprising of four cohort studies, 39-42 90 cross-sectional studies, and one randomised controlled trial (RCT) 43 were included in this review, involving 61,401 HCWs.
Only one cohort study by Leskovic et al. 39 addressed the primary aim of this review. It recruited a mixed population of nurses and healthcare assistants, involving 1009 participants, of which 328 completed both questionnaires before and during COVID-19. Across the participants, the percentage of females was 95.0% and the mean age was 41.3 years. Burnout among HCWs was assessed using MBI scores and compared between before and during COVID-19 to evaluate the impact of the pandemic on HCWs' burnout.
Among the remaining three cohort studies, two studies 40,41 involved only doctors, while the last study 42 involved mixed professions. While all three studies assessed burnout before and during COVID-19 using MBI scores, Baro et al. 40 and Mills et al. 41 used an abbreviated MBI, which is not validated. The cohort study by Kok et al. 42 collected pre-COVID-19 burnout data close to the start of COVID-19 (defined in this review as 1 December 2019). Hence, the data was treated as two cross-sectional datasets for burnout assessment during COVID-19 instead as it was hard to remove the risk of bias with the baseline burnout data from that study. In the pooled data for all four cohort studies, the percentage of females was 88.2% and the mean age was 41.8 years.
The sole RCT 43 identified in this review examined the impact of a mobile application on burnout outcomes among HCWs with mixed professions during COVID-19, comprising an intervention group and a control group. While the study did not match the primary aim of this review, its reported burnout scores were treated as two cross-sectional datasets for analysis. Across its population, the percentage of females was 83.2% and the mean age was 41.4 years.
Burnout outcomes for analysis. This review focused on burnout measured with the MBI. Different definitions of burnout, variations of MBI and MBI cut-off scores defining high EE, high DP, and low PAwere used to assess burnout among HCWs.
Prevalence of burnout based on definitions and MBI cut-off scores. Across the studies, definitions used to categorise HCWs with burnout and HCWs who met the cut-off scores for high EE, high DP, and low PA were different. Table 1 presents the definitions of burnout used in the various studies, and the respective overall reported burnout prevalence among HCWs during COVID-19. Out of 90 cross-sectional studies, those that had missing data or used modified MBI (n studies = 61) were excluded.
Aside from the definitions of burnout, the studies also used different MBI cut-off scores to categorize high EE, high DP, and low PA. Prevalence of burnout based on the MBI variant. Figure 2 shows the box-and-whisker plot of the weighted means for the prevalence of overall burnout, high EE, high DP, and low PA across all studies (n studies = 67), excluding those that used modified MBI (n studies = 28). 40 Prevalence of burnout based on healthcare profession. Figure  3 shows the box-and-whisker plot, while Table 4 provides the quantitative data of the weighted mean prevalence of overall burnout, high EE, high DP, and low PA across studies that used MBI-HSS or MBI-HSS(MP), stratified by profession. Studies with burnout data on non-mixed professions (n studies = 21) and examined specifically doctors (n studies = 11) 44,46,47,50,51,53,56,59,60,62,63 or nurses (n studies = 7) [64][65][66][69][70][71]74 were analysed together in the figure below. Studies that examined other professions 75,76,80 were omitted from the figure due to limited data. As seen from Table 4, the mean burnout prevalence among doctors (45.50%) and nurses (42.50%) were both higher than that reported for the overall population (39.48%) during COVID-19. However, as seen from Figure 3, there were wide ranges reported across the professions, suggesting large heterogeneity across the studies.
Quantitative MBI scores measuring burnout subscales. Out of four cohort studies, only one study by Leskovic et al. 39 matched the review criteria. The study compared MBI-HSS scores in HCWs working in Slovenian nursing homes in rural areas, before COVID-19 in Spring 2013 (n participants = 556), versus during COVID-19 in Spring 2020 (n participants = 781). They reported a significant increase in mean EE from 21.9 to 24.8 (p = .001), but significant increase in mean PA from 42.7 to 48.7 (p = .001). There was no significant change in DP from 8.1 to 8.2 (p = .467). While the increase in EE signalled a worsening of burnout, the increase in PA suggested an improvement.
Two of the remaining cohort studies used modified MBI 40,41 and hence were excluded for analysis. The last cohort study 42 examined burnout scores among HCWs before and during    COVID-19. However, this study was excluded from analysis as the pre-COVID-19 data was collected during the start of the pandemic, hence not considered as a fair comparator. Instead, the data from this cohort study was treated as a cross-sectional dataset to assess burnout during COVID-19.
Excluding studies that used modified MBI to measure burnout, 40 Due to the limited number of studies that used MBI-GS 67,72,73,[88][89][90][91][92] and the difficulties in comparing and relating MBI-GS scores to MBI-HSS or MBI-HSS(MP) scores, burnout data from MBI-GS was not reported.

Assessment of risk of bias of included studies
Among the studies, only one study 43 had low risk of bias, 86 studies had moderate risk of bias (90.5%) and eight studies 40,41,52,61,79,84,100,122 had high risk of bias (8.4%).
Of note, the cohort study by Leskovic et al. 39 had a moderate risk of bias due to the possibility of data integrity concerns in view that the mean score of PA reported exceeded the maximum possible score of 48 based on MBI-HSS. The authors have yet to respond to clarifications on this discrepancy, hence the moderate risk of bias rating.
Most studies included in this review were cross-sectional by design and had no comparator, hence a meta-analysis could not be conducted.

Discussion
This systematic review aimed to evaluate the impact of COVID-19 on the burnout of HCW using MBI. However, only Leskovic et al.'s 39 cohort study addressed the primary aim. The remaining studies only reported burnout data during COVID-19, but not before. This limited our ability to explore the change in burnout levels due to COVID-19. Nevertheless, we could were able to determine the overall burnout prevalence measured using MBI among HCWs during COVID-19.

Summary of evidence
Burnout in overall HCW population. Our review found that the mean prevalence of burnout among HCWs during COVID-19 was 39.95%, with a wide range of 6.00%-90.40% ( Figure  2). Based on burnout data reported by previous studies conducted before COVID-19, this mean prevalence falls within the commonly reported range of 30%-70%. [7][8][9][10][11][12] The wide range in burnout prevalence reported reflects large heterogeneity in burnout measured across the studies, and we are hence unable to evaluate and confidently conclude the impact of COVID-19 on HCW burnout. Moreover, most studies were not designed to compare burnout before and during COVID-19 among HCWs, except for Leskovic et al.'s cohort study. 39 That study reported a significant increase in EE and PA among HCWs, suggesting a worsening of burnout based on EE but a slight improvement in burnout based on PA. Based on this single study, however, we are unable to evaluate the impact of COVID-19 on the burnout of HCWs. More studies adopting the appropriate methodology of having historical comparators would be required to address this research question. However, if historical data of pre-COVID-19 period is absent, it is likely that this research question may stay unresolved. For the studies that publish MBI burnout data during COVID-19, it is imperative that there is subsequent follow-up when the pandemic situation improves to reassess the change in the burnout indices to understand the effects of the pandemic on burnout among HCWs.
Aside from prevalence of burnout, mean MBI scores were also examined (Table 5 and Figure 4). Based on the commonly used MBI cut-off scores for high EE ≥ 27, high DP ≥ 10, and low PA ≤ 33, our results suggest that HCWs may not experience burnout across all subscales. However, the wide range of scores reported suggests that local context and unique setting factors may also be important drivers of the burnout in HCWs beyond COVID-19-related factors.
Burnout stratified by healthcare profession. Differences in job scope between healthcare professions prompted us to stratify our burnout data by profession to gain further insight on the impact of COVID-19. Although it is difficult to conduct conclusive comparisons between professions due to the wide ranges reported across burnout and its subscales based on mean prevalence (Figure 3), average MBI scores were generally comparable across professions. Hence, more studies with largely representative sample populations of HCWs can  minimise heterogeneity in the burnout data gathered and thus benefit future reviews evaluating the impact of COVID-19 on the burnout of HCWs stratified by profession.

Challenges and limitations
The challenges to this systematic review include achieving a consensus regarding burnout measured using different variations of MBI, burnout definitions and MBI cut-off scores.
While several variations of MBI were adopted across the studies, MBI-HSS and MBI-HSS(MP) were most used as they are specifically designed for HCWs working in the human services sector and assess mainly on worker-client interaction. 124 Studies which used modified MBI 40,41,45,48,49,52,54,55,57,58,61,68,[77][78][79][93][94][95][96][97][98][99][100][101][102][103][104][105] were initially included in the review because burnout is a specialized field of research, with only a few studies examining this phenomenon among HCWs during COVID-19. However, these studies could not be used for statistical analyses, as there were a wide range of non-validated scales usedranging from single-item to 21-item measures. This made it difficult to pool the data from these studies and evaluate COVID-19's impact on the burnout of HCWs. While MBI is the most validated tool to measure burnout, there is currently no standardized definition in using the scores obtained to categorize the presence or severity of burnout and its three dimensions. These contribute to the large heterogeneity across studies and limits effective comparisons regarding burnout outcomes. Hence, obtaining a standardized definition of burnout will be helpful in consolidating this research. In this review, we identified six definitions of burnout commonly adopted across studies (Table 1), similar to that adopted in previous studies conducted before 11,31 Looking at the collection of research on this topic, adopting a standardized definition of burnout will provide a common point of reference for comparing burnout data among HCWs before, during and/or after COVID-19. Future studies can adopt the most used definition of burnout presented in this review  which is high EE, high DP, and low PA, for expanding the research in this field.
This systematic review has some limitations. Firstly, some articles may have been missed due to different terms used to describe burnout, thus limiting the comprehensiveness of results presented. Secondly, many studies included had high risk of bias due to non-randomized methods when recruiting participants and the presence of confounders. Additionally, there is likely publication bias as burnout research may not be highly prioritized in an already burnout healthcare system, hence the data may not be well-established in the published literature. As reflected by the wide range of intervals reported for burnout measures, it appears the studies have large heterogeneity. Finally, there is insufficient studies with pre-COVID-19 data to evaluate the impact of COVID-19 on the burnout of HCWs. This highlights the need for studies to have appropriate comparators or longitudinal follow-up, for instance pre-COVID-19 and post-COVID-19, to better understand impact factors of burnout in HCWs.

Conclusion
Only one cohort study evaluated the impact of COVID-19 on the burnout of HCWs. Despite the lack of studies with pre-COVID- 19

Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.

Supplemental material
Supplemental material for this article is available online.