{'fraction_non_alphanumeric': 0.029895666131621187, 'fraction_numerical': 0.012439807383627608, 'mean_word_length': 5.301201011378002, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 2, 'https://': 5, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Evaluation of the reporting quality of clinical practice guidelines on melanoma using the RIGHT checklist

Background The International Reporting Items for Practice Guidelines in Healthcare (RIGHT) instrument was launched in 2016 to improve the reporting of clinical practice guidelines (CPGs). We aimed to systematically evaluate the reporting quality of CPGs on melanoma using RIGHT. Methods We systematically searched electronic databases, guideline databases and medical society websites until November 2020 to identify guidelines for melanoma published since 2018. The reporting quality of included guidelines was assessed by calculating the percentages of the 35 items of the RIGHT checklist that were appropriately reported. We stratified the results by selected characteristics to describe the correlation of these factors with reporting quality. Results A total of 20 guidelines were identified and analyzed. The mean reporting rate was greater than 50% in five of the seven domains of the RIGHT checklist; the remaining two domains (Other information, Review and quality assurance) both had a mean reporting rate of 35.0%. The mean overall reporting rate was 63.7%. No CPG considered equity, feasibility or acceptability of the recommendations (item 14c), and only one CPG described the role of funders (item 18b). Guidelines that reported funding or were published in higher-impact journals tended to have a higher reporting quality, whereas the reporting rate in the one included Chinese-language CPG was low. Conclusions Reporting quality of melanoma CPGs tends to be relatively good. The CPGs developed in China were however an exception. The use of the 2016 RIGHT tool in guideline development should be encouraged to support rigorous and transparent reporting.


Introduction
Melanoma is a malignancy whose incidence and mortality rates across the world are strongly associated with socioeconomic and lifestyle factors (1). In 2018, there were an estimated 290,000 new cases of melanoma and approximately 61,000 deaths caused by melanoma worldwide (2). Once melanoma becomes invasive or spreads, it is life-threatening. In the past decade, increasing knowledge of melanoma has led to significant advances in fighting this disease. After the introduction of the immunotherapy ipilimumab in 2011 (3), patient survival has improved rapidly. Since then, nine new drugs have been approved, including immunotherapy with anti-PD-1/ PD-L1 monoclonal antibodies, and targeted therapy with BRAF inhibitors and MEK inhibitors (4,5). Molecular pathogenesis and clinical imaging technologies have also advanced, resulting in a revolution in the melanoma staging system and patient stratification (6). The new imaging technologies do not only inform prognostic assessment, but also improve clinical outcomes. Surgical resection, chemotherapy and radiotherapy remain relevant, and should also be incorporated into clinical decision making (7). The rapidly expanding therapeutic landscape has also led to rising costs (8). Given the multiple therapeutic options and their high costs, the ways to integrate these approaches in the optimal way need to be assessed.
Many clinical practice guidelines (CPGs) have been developed to provide guidance on the screening, diagnosis, surgical and medical treatment approaches, and further management of melanoma. The existing guidelines also address more specific issues, such as melanoma management in special populations (9), genetic testing for hereditary risk factors (10) and the management of the unique toxicity profiles related to novel therapies (11). CPGs can decrease the variability in healthcare procedures and help to choose the appropriate approach in each specific clinical situation (12). This expectation relies on the assumption that the CPG has been rigorously developed, and that the recommendations are easy to understand and implement. The quality of CPGs, therefore, is of particular concern. To our knowledge, the quality of melanoma guidelines has not yet been systematically assessed.
Explicit and clear reporting is an important aspect of the quality of a CPG. The International Reporting Items for Practice Guidelines in Healthcare (RIGHT) instrument, published in 2016, is a tool for assessing reporting quality of guidelines according to 22 key criteria, grouped into seven domains (13). In this study, we systematically searched for guidelines for melanoma and assessed their reporting quality using the RIGHT tool. Our ultimate aim was to identify weaknesses and strengths in reporting to inform the framework of future development of melanoma guidelines.

Study design
We conducted a systematic review and critical appraisal of the reporting quality of CPGs for melanoma using the RIGHT checklist.

Literature search
We searched Medline (via PubMed), Chinese Biomedical Literature Database (CBM), Wan Fang Database and Chinese National Knowledge Infrastructure (CNKI) from their inception to November 27, 2020, using the following terms: melanoma, nevocarcinoma, melanomatosis, practice guideline, guidance, and recommendation. Our search was restricted to publications in English and Chinese. Additionally, we searched the websites of The National Institute for Health and Care Excellence (NICE, https:// www.nice.org.uk/), The National Comprehensive Cancer Network (NCCN, https://www.nccn.org/), World Health Organization guidelines (WHO, https://www.who.int/ publications/guidelines/year/en/), Scottish Intercollegiate Guidelines Network (SIGN, https://www.sign.ac.uk/ourguidelines/), and Guidelines International Network (GIN, https://guidelines.ebmportal.com), as well as Google Scholar as supplemental sources.

Inclusion criteria and exclusion criteria
CPGs were eligible for inclusion if they focused on screening, testing, diagnosis, treatment, or management of melanoma; the full-text was accessible; and the most recent updated version was published between 2018 and 2020. We excluded records that were translations, summaries or interpretations of guidelines. Drafts and preprints of guidelines were also excluded.

Screening
All records were exported to a bibliographic file and imported into EndNote X8 (Thomson Reuters) for management and screening. Titles and abstracts were screened by AL Zhang and XH Liu to determine their relevance. Final selection was based on full-text screening of all potentially applicable articles and relevant supplementary materials. Ambiguous articles were examined by a third reviewer (QW Zhang). Disagreements were resolved by discussion with a senior third author (XJ Zhang).

Data extraction of guidelines
All data were extracted in duplicate (KF Liu and S Tang) using a standardized electronic form. Disagreements were settled through consensus or, if necessary, consultation with a third reviewer (J Kang). We extracted the following data from each CPG: first author, region/country where the CPG was developed, developers (institution or working group), publication year, publication language, format of publication (peer-reviewed journal, or website only), impact factor (IF) of the journal according to the Science Citation Index (SCI), and the scope/purpose and target population of the CPG.

Reporting quality assessment using RIGHT checklists
The reporting quality of CPGs was evaluated using the RIGHT tool, which includes 22 key items categorized into seven domains. Some key items were divided into two or three sub-items, resulting in a total of 35 items. The seven domains are: basic information (six items), background (eight items), evidence (five items), recommendations (seven items), review and quality assurance (two items), funding and conflicts of interest statements and management (four items), and other information (three items). We rated items as "reported" (relevant information was fully presented), "unreported" (lacked some relevant information), or "not applicable" (not appropriate for evaluating in the specific guideline), based on the protocol of the RIGHT tool. The evaluation method was checked for validity and consistency by two co-authors (YJ Yang and JL Lu), who subsequently independently evaluated the reporting quality of all CPGs. Disagreements were settled by consultation with a member of RIGHT checklist working group (YF Ma).

Statistical analysis
The numbers and proportions of the RIGHT items that were appropriately reported were calculated for each guideline. We calculated the mean reporting rates over each domain of RIGHT as well as overall weighting all 35 items equally. We present the results as means over all CPGs, as well as stratified by the type of organization and country that led the guideline development, the language of publication, the year of publication, the IF of the journal, and funding support.

Identification of guidelines
We identified 865 potentially relevant records in the literature search ( Figure 1). After exclusion of 17 duplicated records, we screened 848 titles and abstracts, and excluded 820 that were not eligible. We retrieved 28 references for full-text assessment. A total of 20 CPGs met the inclusion criteria and were included in the analysis.

Characteristics of selected guidelines
The characteristics of the included 20 CPGs are shown in Table 1. Three CPGs were published in 2018, eight in 2019, and nine in 2020. Most CPGs (n=17) were developed by medical specialty societies, and three by a guideline working group. Eight CPGs were developed in North America, seven in Europe, three in Asia, one in Australia, and one was developed by an international consortium. One CPG reported receiving no funding, nine reported receiving government or society funding, and ten did not report whether they received funding or not. Twelve CPGs were updated versions, six were original versions, and two were adapted from other guidelines. Two CPGs used GRADE to grade the quality of evidence and strength of the recommendations, two CPGs Oxford classification, and three CPGs the Infectious Diseases Society of America-United States Public Health Service Grading System. Only two CPGs were developed following the Appraisal of Guidelines for Research & Evaluation II (AGREE II) framework in their methodology.

Overall analysis of reporting quality
Five of the seven RIGHT domains had a reporting rate of >50% ( Figure 2). The reporting rates were 67.5% for the basic information domain, 81.9% for the background domain, 61.4% for the recommendations domain, and 51.2% for the funding and declaration and management of interests domain. Two domains (other information; review and quality assurance) The flow chart of the selection process.

Records identified from literature databases (n=850)
Records screened for title and abstract (n=848) Full-text articles assessed for eligibility (n=28) Guidelines included in the analysis (n=20) Records excluded as duplicates (n=17) Records excluded (n=820) Excluded for full-text articles, with reasons: Records identified from other sources (n=15) had a reporting proportion of 35.0%. The overall reporting rates ranged between 28.6% and 82.9% across the guidelines. The mean overall reporting rate over all guidelines was 63.7%, and only two CPGs had rates above 80% ( Figure 3). All CPGs reported the items 1a and 3 in the basic information domain, items 7a and 7b about the target population in the background domain, and items 13a and 13b in the recommendations domain ( Table 2). No CPG reported item 14c (recommendations domain), only one CPG described the role of funders (item 18b, funding source and declaration and management of interests domain), and two CPGs indicated if and how quality assurance was evaluated (item 17, review and quality assurance domain).

Stratified analyses
The mean overall reporting proportion improved slightly over time, being 60.0% in guidelines published in 2018, 61.4% in 2019, and 67.3% in 2020 ( Table 3). Guidelines published in Chinese had a reporting rate of 37.1%; for guidelines published in English the reporting rate was 65.3%. Guidelines that reported their funding sources had a higher reporting rate (71.1%) than those that either did not report funding, or reported receiving no funding (57.9%).

Discussion
To our knowledge, this is the first analysis of reporting quality specifically assessing CPGs for melanoma. We identified several patterns in the reporting quality of CPGs for melanoma. First, with the exception of the two guidelines developed in China, the quality of reporting was good, with reporting rates of >50%. Second, we identified two guidelines, one from the USA and one from Australia, which had clearly the highest reporting rates and which therefore could serve as a template for future guideline development. Third, the domains other information and review and quality assurance had the poorest reporting quality. Fourth, no guideline described whether developers had considered factors such as equity, feasibility and acceptability in formulating the statements, which can be expected to affect the dissemination and implementation of the recommendations. These findings are partly consistent with the recent findings about reporting quality of CPGs for assisted reproductive technology (31) and idiopathic pulmonary fibrosis (32), as well of the WHO guidelines (33). The domains basic information and background had a relatively high reporting rate across all guidelines. However, the publication year and a summary of the recommendations tended to be poorly reported. Of the top 50% of guidelines ranked by overall reporting proportions, none reported the year of publication in the title, and 70% of them did not have a summary.
Most guidelines included in our review had deficiencies in reporting review and quality assurance process. Items pertaining to external review, peer reviewers, review process and management of the feedback were poorly reported. One potential explanation for this finding is that appropriate consideration of quality assurance reflects a greater awareness of developers on the importance of guideline reporting.
Guidelines with funding support had a high overall reporting quality than those not declaring or having funding. Funding is of great importance, because the development, maintenance, effective dissemination and implementation of guidelines is an expensive and laborintensive endeavor (34). CPG panels with financial support may confer quality benefits. However, few guidelines reported the sources of funding for the different stages of the development, dissemination and implementation of the guideline and recommendations. The results suggest a need for greater transparency and rigor in the role of funders.
The included Chinese CPGs had an overall reporting quality clearly below the average. Poor quality of Chinese CPGs has also been demonstrated in other guideline reviews (35). One potential explanation for this finding is that Chinese-language journals do not have as high a requirement for guidelines as journals indexed by SCI, although this may also reflect a lack of awareness on the importance of guideline reporting among Chinese authors.
Most CPGs for melanoma used formal grading systems to assess the quality of evidence and strength of recommendations. Consequently, items in the evidence and recommendations domains were reported at relatively high rates. Of note, only few guidelines considered the costs and resource implications, and no guideline considered equity, feasibility or acceptability when formulating the recommendations. A previous study has consistently demonstrated inadequacy of cost analysis in guideline development, even for the most highly cited CPGs (36). CPGs should be adapted to different settings by considering local resources and feasibility. Even evidence-based recommendations may not be applicable to all populations, or they may not be implemented due to their high costs.
We acknowledge that our study has several limitations. First, we did not perform a formal statistical assessment. Our descriptive stratified analyses revealed correlations between reporting quality and some characteristics, but we could not assess the influence of potential confounding factors. Second, we only included English-and Chineselanguage CPGs. We however expect the analysis to reflect the reporting quality of melanomas relatively well, as it covers CPGs developed by the key oncological societies, such as NCCN, American Society of Clinical Oncology (ASCO), Chinese Society of Clinical Oncology (CSCO)      Systematic reviews 11a Indicate whether the guideline is based on new systematic reviews done specifically for this guideline or whether existing systematic reviews were used If the guideline developers used existing systematic reviews, reference these and describe how those reviews were identified and assessed (provide the search strategies and the selection criteria, and describe how the risk of bias was evaluated) and whether they were updated

Questions to be further discussed and considered
Question 1: What impact do you think the low reporting quality of clinical practice guidelines on melanoma will have on clinicians and clinical practices? Expert opinion: Dr. Sanjiv S. Agarwala Unfortunately, clinical practice guidelines are not reported adequately by most centers around the world. However, their importance is unquestionable, and efforts need to be made to improve and encourage participation in such projects. The low reporting quality as demonstrated in this paper will likely still have an impact on clinical practices and on clinicians.
Even low-quality data on guidelines are still important and can help clinicians choose the appropriate therapy for their patients. Busy clinicians and practices are always looking for data that can help them improve outcomes and reduce costs and guidelines assist in that direction. The most effective way to develop high-quality guidelines is to involve more clinicians and practices in the data collection process. As this is usually time-consuming and not the highest priority for a clinician, this will take encouragement and assistance with resources. Most practices are busy with taking care of patients and will need financial incentives and the ability to hire dedicated individuals who can spend their time in collecting and reporting the data.

Expert opinion: Dr. Dirk Schadendorf
High-quality is important but similarly usability and accessibility of guidelines is of critical importance alsofurthermore language used to understand by non-specialists and potentially also for patients in lay-language help to spread the guidelines -because penetration of guidelines into all levels of clinical management routines is rarely achieved. To unfold the entire power of guidelines quality aspects, but also practical usage at all levels will finally determine the clinical effect on patient outcome.

Question 3: How do you think conflicts of interest in the guidelines should be handled? Expert opinion: Dr. Sanjiv S. Agarwala
A clear and accurate process for self-reporting of COI is the first step. Random audits of the reporting should be conducted. National data bases for collection of payments to physicians such as the "sunshine act" in the USA need to be established so that there is transparency and accuracy of the data.

Expert opinion: Dr. Dirk Schadendorf
Conflict of interest should clear and fully transparent.

Conclusions
In conclusion, our findings show that CPGs for melanoma tended to be appropriately reported, except two guidelines developed in China. The quality of reporting has improved over time, possibly as a result of the increasing standards of CPG development in recent years. However, the reporting of some aspects, such as funding sources and the role of the funder, still need to be improved. The use of RIGHT tool when developing and updating guidelines ensures rigorous, complete and transparent reporting.