{'fraction_non_alphanumeric': 0.03541370641600175, 'fraction_numerical': 0.012788282872445076, 'mean_word_length': 4.854126679462572, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Assessment of the quality of systematic reviews on COVID‐19: A comparative study of previous coronavirus outbreaks

Abstract Several systematic reviews (SRs) have been conducted on the COVID‐19 outbreak, which together with the SRs on previous coronavirus outbreaks, form important sources of evidence for clinical decision and policy making. Here, we investigated the methodological quality of SRs on COVID‐19, severe acute respiratory syndrome (SARS), and Middle East respiratory syndrome (MERS). Online searches were performed to obtain SRs on COVID‐19, SARS, and MERS. The methodological quality of the included SRs was assessed using the AMSTAR‐2 tool. Descriptive statistics were used to present the data. In total, of 49 SRs that were finally included in our study, 17, 16, and 16 SRs were specifically on COVID‐19, MERS, and SARS, respectively. The growth rate of SRs on COVID‐19 was the highest (4.54/month) presently. Of the included SRs, 6, 12, and 31 SRs were of moderate, low, and critically low quality, respectively. SRs on SARS showed the optimum quality among the SRs on the three diseases. Subgroup analyses showed that the SR topic (P < .001), the involvement of a methodologist (P < .001), and funding support (P = .046) were significantly associated with the methodological quality of the SR. According to the adherence scores, adherence to AMSTAR‐2 items sequentially decreased in SRs on SARS, MERS, and COVID‐19. The methodological quality of most SRs on coronavirus outbreaks is unsatisfactory, and those on COVID‐19 have higher risks of poor quality, despite the rapid actions taken to conduct SRs. The quality of SRs should be improved in the future. Readers must exercise caution in accepting and using the results of these SRs.

address a specific research question. Practitioners mainly rely on SRs for evidence to provide evidence-based recommendations. Before the COVID-19 pandemic, outbreaks of severe acute respiratory syndrome (SARS) and the Middle East respiratory syndrome (MERS) were observed in 2003 and 2012, respectively; substantial numbers of SRs have been conducted on these outbreaks. The causative agents of these three outbreaks belong to the same genus of coronavirus (CoV), and SARS-CoV-2 shares a 79.5% sequence identity with SARS-CoV. Thus, the available SRs on SARS and MERS are helpful in guiding COVID-19 management. However, the quality of these SRs was unclear. In addition, besides these existing SRs, we also found that a large number of newly-conducted SRs with or without meta-analyses have been published rapidly, shortly after the COVID-19 outbreak, and the number of such SRs keeps rising. As we know, at the preliminary stage of any public emergency, primary studies are usually lacking, and most studies are observational. Thus, it is not easy to conduct an SR. These recent SRs on COVID-19 are more likely to have the potential methodological flaws. Therefore, we hypothesized that there may be a difference in methodological quality between these SRs on COVID-19 and previous SRs on SARS and MERS.
Up to now, no studies appraising the methodological quality of the SRs on COVID-19 and previous MERS and SARS outbreaks are available. In this comparative study, we investigated the present status of conducting SRs on COVID-19, MERS, and SARS, appraised the methodological quality of these SRs using the a measurement tool to assess systematic reviews (AMSTAR 2), and performed a preliminary examination of the potential risk factors associated with the quality of SRs, with the aim of providing suggestions from the aspects of methodological quality for conducting and using SRs during the COVID-19 pandemic.

| Data sources and search
We searched for eligible SRs (and/or meta-analyses) in both English

| Inclusion and exclusion criteria
We included all SRs which discussed one of the following diseases: COVID-19, SARS, and MERS. SRs were identified based on the following criteria: (a) label of "systematic review" in the title, abstract, or full text; and (b) a literature search was performed. There were no restrictions on the SR topic. SRs with or without meta-analysis were both eligible. Only SRs published in academic journals were included.
Studies were excluded based on the following criteria: (a) letters, editorials, and expert opinions; case reports and literature reviews; and other narrative reviews; (b) abstract only or data unavailable; or (c) studies in non-English or non-Chinese languages. When a duplicate or updated publications were identified, only the most recent one was included.

| Study selection and data extraction
Two reviewers independently screened the titles and abstracts of all articles identified in the initial search and then checked the full texts of potentially eligible studies. Any disagreement was resolved by discussion or consultation with a third reviewer. The same reviewers used a predesigned table to collect the following information on the included SRs: first author, publication year, the growth rate of SRs, region or country in which the study was performed, study topic

| Assessment of the methodological quality of the included SRs
The methodological quality of the included SRs was independently assessed by two well-trained reviewers using the AMSTAR-2 tool. 5,6 Any disagreement between the two reviewers was resolved by discussion, and in case of persistent disagreement, a resolution was arrived at by consulting with a third person.
The AMSTAR-2 is a critical appraisal tool for SRs of randomized control trials and/or observational studies. 6 The AMSTAR-2 contains 16 items, of which seven are critical domains (Table 1). A questionspecific point scale ("Yes, No;" "Yes, Partial yes, No;" or "Yes, No, No meta-analysis") was used to score each item. Based on the degree of weaknesses detected in critical and noncritical items, the AMSTAR-2 classifies the overall confidence in the results of the SRs into four levels: high, moderate, low, and critically low. We graded each included SR on the AMSTAR-2 official website and generated the overall confidence using their online calculator. In addition, to 884 | compare adherence to items among SRs, a total score was calculated for each SR according to the response to each item, that is, "Yes," "Partial yes," "No meta-analysis," and "No" were scored as "+1," "+0.5," "0," and "−1," respectively.

| Data analysis
Descriptive statistics were used to present the characteristics and methodological quality of the included SRs. Categorical variables were expressed as frequencies and percentages. The response to each AMSTAR-2 item was recorded and tabulated for all included SRs. Overall adherence to the 16 items were analysed with a percent stacked bar chart. The adherence scores in each disease group were presented as mean ± standard deviation (SD). Subgroup analysis was performed to compare the differences in quality among subgroups with different study characteristics. Statistical significance was tested using the Kruskal-Wallis rank test and two-sample Mann-Whitney U test. P < .05 was considered to be statistically significant, and all P values were two-sided. Excel 2019 and SPSS V25 were used for all data management and analyses.   were supported by at least one funding source ( were of critically low quality. Not a single SR was rated as having high

| Factors related to SR quality
The potential factors affecting the quality of SRs were investigated in all samples. Subgroup analyses of different variables showed that the topic of the SR, involvement of a methodologist, and funding support were significantly associated with the methodological quality of the SRs (  Details of the assessment of each item are described in the Table S1.

| DISCUSSION
SRs provide the highest level of evidence in evidence-based medicine, and they are an important source of information for clinical practitioners and policy makers. However, due to the relatively low requirements and costs of conducting SRs, the number of SRs published in various fields is quite high. The increase in quantity has also brought concerns about the quality of SRs. 56 In this study, we found that a large number of SRs were conducted de novo after the  For unknown emerging diseases, the scarcity of large samples and high-quality original reports are major challenges in scientific research. In particular, in the early stages of disease outbreaks, evidence is mostly acquired from observational studies, 58

| CONCLUSIONS
SRs specific to COVID-19, SARS, and MERS are being heavily relied upon during the current COVID-19 pandemic. The methodological F I G U R E 2 Adherence to each item in AMSTAR-2. AMSTAR, a measurement tool to assess systematic reviews; MA, meta-analysis quality of most SRs is unsatisfactory, and those on COVID-19 have higher risks of poor quality, despite the rapid actions taken to conduct SRs. Teams that may want to conduct a SR should focus on the study design and focus on improving the quality of the SR. SR findings should be used more cautiously, and it is not advisable that users accept the results of a single SR without critical appraisal.