{'fraction_non_alphanumeric': 0.04129753305165599, 'fraction_numerical': 0.0239880059970015, 'mean_word_length': 5.34717416378316, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The development of clinical guidelines in China: insights from a national survey

Background Previous research suggests that the quality of clinical guidelines (CGs) in China is suboptimal. However, little is known about the methodology that CGs follow. We conducted a national survey of methods used by Chinese CG developers for CG development, adaptation, and updating. Methods We used a previously piloted questionnaire based on methodologies of CG development, adaptation, and updating, which was distributed during September–November 2020 to 114 organizations identified from published Chinese CGs (searched 2017–2020), recommended by Chinese CG developers, and recommended by clinical discipline experts. Results We collected 48 completed questionnaires (42.1% response). Most organizations developed CGs based on scientific evidence (89.6%), existing CGs (75%), or expert experience and opinion (64.6%). Only a few organizations had a specific CG development division (6.3%), a CG monitoring plan (on clinicians 33.3%; on patients 18.8%), funding (33.3%), or a conflict-of-interest (COI) management policy (23.4%). Thirty (62.5%) organizations reported using a CG development methodology handbook, from international organizations (14/30, 46.7%), methodology or evaluation resources (3/30, 10.0%), expert experience and opinion (3/30, 10.0%), or in-house handbooks (3/30, 10.0%). One organization followed a published adaptation methodology. Thirty-eight organizations (88.4%) reported de novo CG development: 21 (55.3%) formed a CG working group, and 29 (76.3%) evaluated the quality of evidence (21 [72.4%] using a methodological tool). Nineteen organizations (52.8%) reported CG adaptation: three (31.6%) had an adaptation working group, and 12 (63.2%) evaluated the quality of source CGs (2 (16.7%) using the AGREE II instrument). Thirty-three organizations (68.8%) updated their CGs, seven (17.5%) using a formal updating process. Conclusions Our study describes how CGs are developed in a middle-income country like China. To ensure better healthcare, there is still an important need for improvement in the development, adaptation, and updating of CG in China. Supplementary Information The online version contains supplementary material available at 10.1186/s12961-021-00799-7.

In low-and middle-income countries, the lack of appropriately developed CGs to assist healthcare practice is resulting in suboptimal clinical practice [7,8]. Although reviews of CGs show that their methodological quality has improved in the past decade [9,10], in China the quality of CGs continues to be inferior [11][12][13][14][15][16]. The evidence shows that Chinese CG quality, as assessed by the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument [17], is scored at under 30% in most domains [16]. A 2015 study of 109 Chinese CGs reported that only a handful were developed based on research evidence (16; 14.7%), while even fewer critically assessed the certainty of evidence (14; 12.8%) or the strength of recommendations (13; 11.9%) [12].
Empirical evidence shows that China lacks high-quality clinical and epidemiological studies or other types of studies as evidence-based resources [18], which may hinder the adequate updating of Chinese CGs or adaptation for local use. Factors that could influence recommendations or informed decision-making, including resources, cost, feasibility, applicability, and equity, are seldom considered in Chinese CG development processes [11,14,15,19]. Lack of proper incorporation of cost and other considerations potentially hinders adherence to Chinese CGs, but may also contribute to the documented tense relationship between doctors and patients [20]. Furthermore, the development process underlying some Chinese CGs based on existing CGs is also unclear. While previous evidence shows that many international CGs have been used to develop Chinese CGs [12]-for example, the National Comprehensive Cancer Network (NCCN) guidelines or National Institute for Health and Care Excellence (NICE) guidelines-how those source CGs were evaluated and adapted is poorly reported [21].
One important challenge, in terms of improving CG quality in China is the fact that little information is available on the methodology used by developers. To gain more knowledge on this, we conducted a national survey to collect data on Chinese CG development methods and to understand how CGs are developed, adapted, and updated, therefore providing the basis for future improvements in guideline quality in China.

Aim
This was a cross-sectional online national survey to better understand how CGs are developed, adapted, and updated in China.

Participants
Participants in our survey were key informants and experts affiliated with CG development organizations and expert committees that have developed Chinese CGs in the past 3 years. We adopted a purposive sampling method to recruit participants [22], identified as follows: (1) corresponding contacts of 74 affiliated CG development organizations extracted from 171 Chinese CGs published between January 2017 and February 2020, and retrieved from a literature search in the China National Knowledge Infrastructure database; (2) recommendations from Chinese CG developers; and (3) recommendations from Chinese clinical discipline experts. If initial contacts were not eligible for participation in the survey, they were asked to recommend an eligible person from their organization. The selection procedure is described in Additional file 1: Appendix 1.

Questionnaire
We developed a self-administered questionnaire based on several methodological and evaluation resources, including AGREE II [17], Grading of Recommendations Assessment, Development and Evaluation (GRADE) [23], GRADE Evidence to Decision (EtD) frameworks [24], Resource Toolkit for Guideline Adaptation (ADAPTE) [6], and the Checklist for the Reporting of Updated Guidelines (CheckUp) [25]. The questionnaire was drafted by one author (YS) and was subsequently reviewed and modified by two other authors (YZ, PAC). The Chinese version of the questionnaire, also available in English (Additional file 1: Appendix 2), was circulated to the contacts in the participating organizations.
The questionnaire consisted of 45 items in five sections: characteristics of the organization (10 questions), de novo CG development (13 questions), CG adaptation (16 questions), CG updating and monitoring (3 questions), and conflict-of-interest (COI) management and funding (3 questions). A free-text box in 33 items collected additional information and comments.

Survey
We used online software (http:// www. wjx. cn) to design the questionnaire and collect responses. The questionnaire adopted a follow-up question format (only participants who answered "yes" needed to answer further questions) [26] and was piloted with four organizations (one national and three international). We refined the survey based on the feedback from pilot testing, which suggested creating follow-up questions and modifying response categories for optimal understanding and response efficiency. We invited participants through email or WeChat message and provided the following information: (1) a description of the study, (2) the purpose of the survey, (3) the main content of the questionnaire, and (4) instructions on completing the questionnaire. We sent two email reminders a month after delivering the invitation and, where possible, reminded potential participants through a WeChat message. On receiving consent from the participants, we sent the survey link by email or WeChat between July and November 2020 and followed up with up to three email or WeChat reminders.

Analysis
Descriptive statistics were used to analyse the study data. Absolute frequencies and proportions were calculated for all responses. Depending on the methodology used by organizations, we stratified CG development as de novo or adaptation. CGs used for adaptation purposes are referred to as "source CGs", and recommendations from source CGs are referred to as "source recommendations". We hypothesized that using a CG development methodology handbook would be associated with the rigour of the guideline development process. The association was determined by Pearson's chi-square test or Fisher's exact test (alpha was set at 0.05). Data were analysed using SPSS version 23.0 statistical software (IBM Corp., Armonk, NY, USA). For qualitative data, one author (YS) coded the data and extracted themes related to CG de novo development or adaptation [27], and another author (JL) double-checked the codes and the corresponding quotations. The most relevant topics raised by respondents in free-text areas of the questionnaire were selected on the basis of consensus among the three authors (YS, YZ, JL).

Results
A total of 114 Chinese CG development organizations and expert committees were contacted by email and WeChat. Responses were received from 55 CG development organizations. After three reminders, we obtained 48 complete responses (42.1% response rate) ( Fig. 1) (Additional file 1: Appendix 3).

Organization characteristics
The organizations, profiled in Table 1, represent six regional economic divisions, 13 provinces, and 13 clinical disciplines as per the Subject Classification of the People's Republic of China [28].  Organizations that reported using a CG development methodology handbook were more likely to use a formal decision-making process (Fisher's exact test; p = 0.009) (Additional file 1: Appendix 4). When formulating recommendations, most organizations reported considering the balance between benefits and harms (81.6%), patient values and preferences (86.8%), cost and resources (86.8%), and other factors (81.6%) (e.g., equity, acceptability, and feasibility). The basis for formulating recommendations varied from expert opinion to the use of research evidence ( Table 2, Fig. 2). The reasons for not considering specific aspects were lack of knowledge or expertise.

CG adaptation
Nineteen of 36 organizations developing CGs through guideline adaptation (52.8%) reported a CG adaptation process (Table 3). Six organizations (31.6%) had an adaptation working group, mainly composed of clinicians (83.3%) and methodologists (83.3%). Most organizations conducted a systematic search to retrieve source CGs (84.2%) and conducted an external review (94.7%). About one in five organizations that conducted a systematic search ultimately did not implement a rigorous search strategy. Eligibility criteria were applied to selecting source CGs by 13 organizations (68.4%), with those who used a CG development methodology handbook being more likely to use a formal eligibility procedure (Fisher's exact test; p = 0.007) (Additional file 1: Appendix 4).
Over 60% of organizations assessed the source CGs for quality (63.2%), currency (100%), content (73.7%), and   (1) analysing the reason for an inconsistency, (2) selecting recommendations from prioritized source CG or based on the applicability of the recommendations to the target setting, and (3) discussion among experts (Fig. 3).
In relation to contextualization, most organizations took into consideration differences between the target setting and the source CG setting, including 16 (84.2%) population differences, 14 (73.7%) health system differences, and 12 (63.2%) clinical practice differences. Approaches to contextualizing source CG recommendations included (1) analysing the reason for differences, (2) supplementing with local evidence, (3) considering expert opinion, and (4) modifying recommendations according to the target context (Fig. 3). In the case that differences could not be solved, reporting differences was considered.
Most organizations reported considering patient values and preferences (94.7%), cost and resources (94.7%), constraints or barriers for implementation (84.2%), and other factors (89.5%). As with de novo CG development, the basis for formulating recommendations varied from expert opinion to considering research evidence ( Table 3). The reasons for not considering specific aspects were lack of knowledge or expertise.

CG updating and plans to investigate adherence
Thirty-three of 48 (68.8%) organizations reported having an updating strategy for their CGs, with seven of them (17.5%) confirming a formal updating process (Table 4). Around 60% of the organizations reported an updating frequency of 3-5 years for their CGs. Plans for investigating clinician adherence and target user adherence to CGs were reported by 16 (33.3%) and nine (18.8%) of 48 organizations, respectively.

Main findings
Our study describes the current CG development process in China, including de novo development, as well as adaptation and updating practices. While CG development in China is broadly in line with international standards, the methods used for specific steps tend to be both variable and informal. CG development is based on varied sources of CG development methodology handbooks and even expert experience and opinion; many developers perform only informal quality assessment of evidence or of source CGs; few organizations have specific CG development divisions, multiple stakeholder engagement, formal updating systems, a COI policy, or funding to support CG development. Similarly, standard methods are not used to adapt source CGs, even though CGs have been adapted for many years in China.

Methods (yes responses) n (%)
The institution has a formal CG adaptation working group 6 (31.6) Evidence is retrieved using systematic searching 16

Our study in the context of previous research
Our findings, compared to those of previous quality assessment studies, show that the rigour of CG development in China is gradually improving. Zhou et al.
(2020), for instance, found that CGs published after 2014 were of significantly higher quality than older CGs [30]. Similarly, a quality assessment of Chinese CGs by Wang et al. [31], published in 2020, reported "rigour of development" scores for CGs published specifically in 2018-2019 that were higher (65.1%) than the overall average median score of below 50%. A new series regarding the development process of evidence-based medicine and clinical guidelines in China published in the Journal of Clinical Epidemiology is also in line with our study findings [32]. Unlike quality assessment studies, our survey identified the methodologies that Chinese CG developers follow, which is not limited to what is reported. The reporting of Chinese CGs is very suboptimal as assessed by the Reporting Items of Practice Guidelines in Healthcare (RIGHT) statement [33][34][35]. Considering that the completeness of reporting impacts quality assessment results for CGs, the assessment scores based on AGREE II are likely to be lower, thereby underestimating the methodological quality of Chinese CGs. Moreover, around 2 years is needed to develop a CG; hence, previous assessment studies reporting poor quality in the AGREE II "rigour of development" domain with the last search date around 2019 or earlier may reflect CG development in or before 2017 [30,31].
Although the rigour of CG development in China is improving, the methods used vary widely. More than 30% of Chinese CG development organizations in our study did not follow any handbooks or guidance on developing CGs, and the handbooks they used were not only standards from different international organizations, but also methodological tools or expert experience and opinion. Given that evidence rating systems and decision-making procedures vary across international organizations, such discrepancies introduce variability in the Chinese CG development process. NCCN, for instance, uses a different evidence rating system from that used by WHO [36], while NICE also has its own decision-making procedure [37]. In addition, the methods used for specific steps, such as assessing the limitations of evidence or the quality of source CGs, tend to be informal.
However, as was reported by a previous study [11], most Chinese CG development organizations do not have a specific division or group for CG development; this makes our findings regarding inconsistent CG development methodology handbook use and lack of quality assurance monitoring less surprising. The funding sources for CG development point to the involvement of industry funding and, therefore, of COIs. Without proper COI management policies, the evidence-based framework and credibility of CGs is inevitably hampered [38,39]. Furthermore, few CG organizations have formal updating or adherence monitoring procedures in place. Although around 20% of recommendations become outdated within 3 years, only 15.2% of organizations update their CGs within this period of time [40]. Another area of concern is that most organizations mainly rely on clinicians and so lack participation by other stakeholders, such as patient representatives and policy-makers.

Methods (yes responses) No. (%)
Updating (n = 48) The institution has a CG updating strategy 33 (68.8) The institution has a formal CG updating procedure 7 (17.5) Monitoring (n = 48) The institution has a plan to check adherence by clinicians 16 (33.3) The institution has a plan to check adherence by target users 9 (18.8)

Specific methods (open-ended responses) No. (%)
Updating frequency (n = 33)  Stakeholder engagement is essential for improving CG recommendation uptake and implementation, which should be considered during the CG development process [4,41]. A lack of stakeholder engagement may lead to controversy and uncertainty, thereby hindering CG implementation [42]. We found that 75% of Chinese CG organizations developed CGs by adapting source CGs, which highlights the widespread use of CG adaptation in China. However, precisely how CG adaptation methods are used is unclear. Of the CG organizations in our study that adapted source CGs, only half reported their adaptation process, and hardly any mentioned following a published adaptation methodology. In addition, as happens with de novo CG development, CG adaptation is informal and lacks monitoring. Only six CG organizations in our study had created an adaptation working group, and only two mentioned having used a validated tool to evaluate the quality of source CGs. Since the quality of adapted CG relies mainly on the source CG, this informality undoubtedly contributes to the low quality of Chinese CGs.

Limitations and strengths
Our study has some limitations. First, the response rate was relatively low, despite sending two reminders and contacting potential participants using different approaches. However, our sample included 48 Chinese CG development representatives of 13 clinical disciplines and 13 provinces. We did not explore CG development on the basis of consensus, which is yet to be studied and understood.
There are several strengths of our study. First, the survey format with follow-up questions allows us to describe in depth the specific methods used in China and to explore the underlying reasons for the low-level quality of Chinese CGs. Additionally, our study comprehensively describes the CG development process in one middle-income country, including CG de novo development, adaptation, and updating process, which contributes to the improvement of the CG development process as a whole in China. Furthermore, we designed the study questionnaire following international standards and piloted it with both national and international organizations. This allows our methods to provide more reference value to other countries with similar issues.

Implications for practice and research
CG development in China needs to be standardized. A good CG development process requires a multidisciplinary working group, a rigorous methodology, sufficient and independent funding, sound COI management, and a monitoring and updating system [1]. Stakeholder engagement should be emphasized in the development process of Chinese CGs to ensure that guideline topics are relevant and prioritized and that other factors like acceptability and feasibility are adequately considered, thus facilitating policy-maker adoption of recommendations into policy and practice [43][44][45]. In addition, sufficient nonprofit public funding and strict COI management strategies should be ensured for CG development, to reduce the potential COI impact on health-related decision-making and clinical practice. Medical associations and government institutions need to assume responsibility for CG monitoring and quality assurance, thereby ensuring the proper implementation of formal development and adaptation methodologies for CGs. CG developers in China need to collaborate closely in standardizing and improving the rigour of CG development, for example, by implementing a standard CG development methodology/handbook and following published reporting guidance such as the RIGHT statement for de novo CGs or CheckUp for updated CGs [25,46]. Future practices need to build on those aspects so as to improve the quality and reliability of Chinese CGs, and therefore improve healthcare nationwide.
While CG adaptation is an efficient way to develop contextualized recommendations, adapted CGs will only benefit from the quality of source CGs by implementing a rigorous adaptation process [6,47,48]. Of the quality published adaptation methodologies available [49], Chinese CG developers could adopt and validate an optimal methodology applicable to the national context. Future research could therefore focus on exploring efficient and rigorous adaptation methods that ensure CG quality and also improve CG implementation.

Conclusions
CG development and adaptation methodologies, including for updating, as used in China tend to be variable and informal, and so need to be standardized. CG development in general is poorly managed and monitored. Greater effort and more funds need to be invested in improving the quality of Chinese CGs so as to ensure better healthcare.