{'fraction_non_alphanumeric': 0.04116237827074453, 'fraction_numerical': 0.011667340779689494, 'mean_word_length': 5.477629987908101, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Value in oncology from multi-criteria decision analysis: A systematic review

Objective: This study aims for three purposes: a) review the published MCDAs in oncology to identify the criteria considered valuable by the stakeholders, b) evaluate the adherence to best practice recommendations, and c) Propose a value tree, based on the findings of the systematic review, reflecting the most important criteria for the different stakeholders. Material and Methods: This systematic review consisted of the following phases: identification, screening, eligibility assessment, assessment of adherence to best practices, and extraction of the information. The identification was conducted in PUBMED, EMBASE, EBSCO, SCIENCE DIRECT, SCOPUS, LILACS, and Web of Science including records from January 1st, 1990, to February 28th, 2021. The adherences to best practices in MCDA were evaluated. A new value tree was made. Results: Thirteen articles were included. Colon, breast, and haematological cancer were the most frequently evaluated (n=10, 69.2%). Physicians and patients were the most representative participants. The value measurement approach was the most used (n=11, 84.6%). The overall adherence rate to the recommendations was 77,3%. One hundred ninety-five criteria were identified. The relevant criteria for all stakeholders were “Improvement clinical efficacy" (24.5%), "Severity of disease" (13.5%), and "Improvement of safety & tolerability." (10.3%). The physicians valued "Improvement clinical efficacy" (28.4%), "Severity of disease" (10.5%), and "Improvement of safety & tolerability." (8.4%) The most relevant criterion for the patients was "Severity of disease" (34.4%), "Improvement of clinical efficacy." (24.1%), and "Improvement of perceived health status" (13,8%). The significant criteria for administrative and academics were "Innovativeness of intervention" (37%) and "Improvement clinical efficacy" (14.8%). Conclusion: The number of MCDAs in oncology is scarce and with moderate adherence to best practice recommendations. A value tree based on relevant criteria was proposed.


INTRODUCTION
The health systems are incorporating a value-based healthcare delivery paradigm. In this context, value has been defined as "the relationship between resources, outcomes, and context." (1) and value-based healthcare as "the use and proportionate allocation of resources, prioritizing the results and experiences in patients' lives in a fair, and context-based way (1,2). These definitions highlight the efficient use of resources in creating worthy health-related experiences for the patients as a feature of this paradigm. The efficient allocation of resources and the prioritization of the patients' experiences are affected by distinct factors. Growing demand for health services by empowered population, demographic changes in the population, the vertiginous development of new medical technologies, and the population's expectations for adopting them impose budgetary pressure on the health system (3). Although increased health resources might improve patient's reported outcomes, this is not always the case (4,5). This discrepancy might be explained by unjustified variations in care between patients (i.e., either through excessive or insufficient use of resources), adoption of new and expensive technologies without meaningful clinical impact, and the exclusion of patient's values and expectancies (1,(6)(7)(8). Under this scenario, a value-based paradigm might identify meaningful approaches that improve efficiency, patient satisfaction, and equity in healthcare (1,2).
Identifying these approaches is essential for all health situations and people but is particularly relevant for cancer patients. Several particularities support this relevance in this population, such as the significant increase in therapeutic alternatives (e.g., the increasing use of monoclonal antibodies for different cancers) and the need to evaluate the particular risk-benefit profile according to numerous clinical stages and molecular typification (e.g., different prognosis and chemotherapy protocols in case of presence of HER-2 in breast cancer) (9,10). Other factors include the high impact of the adverse effects on the patient's quality of life and the need for treatment by multidisciplinary teams (e.g., surgeons, clinical oncologists, palliative care) (9)(10)(11)(12)(13). These unique attributes might explain the high cost for the health system and the patients affected by this condition (14,15). These considerations underline the need to identify what is considered valuable by different stakeholders to direct resources towards it.
There have been attempts to respond to the need for value creation in oncology. For this, different organizations such as the American Society of Clinical Oncology (ASCO), National Comprehensive Cancer Network (NCCN), Institute for clinical and economic review (ICER) have created tools called "value frameworks" (16-18). These clinical instruments allow the physician and patient to jointly assess the value of different therapeutic alternatives based on context. For this, there is a broad and transparent exposure of the entire range of treatment possibilities in several aspects (e.g., clinical benefit, economic implications, toxicity, among others) to reach a consensual decision based on clinical recommendations and the patient's priorities and preferences (19,20) Despite being such useful and innovative tools, value frameworks have significant limitations to their application. Firstly, these tools are applicable for choosing therapeutic alternatives exclusively, and they do not consider other interventions (i.e., preventive, diagnostics, or rehabilitation interventions). Secondly, these frameworks are limited to using clinical criteria to evaluate the different alternatives without including subjective criteria that may be important for decision-making by the patient (19) thirdly, these instruments have only clinical results without considering other aspects of the patients' experience or their perspective. Finally, there are conceptual and methodological inconsistencies due to the lack of criteria and well-established procedures to address uncertainty. (1) The limitations above have led to exploring alternative methods to support the value-based decision-making process. In this context, multi-criteria decision analysis (MCDA) has been gradually introduced as an alternative for health decision-making (19). This analytical tool has been applied in a vast field of health interventions (21). This approach allows multiple interest groups to participate, considering each one's considerations and preferences. Besides, it includes different criteria, not only clinical ones, including other interests of the participants (e.g., subjective ones). It also broadens its focus, including not only therapeutic alternatives but also diagnostic and rehabilitation ones.
By presenting all the criteria established for a specific alternative, this tool allows the participants to weigh the criteria, making more just and practical decision-making. Additionally, this type of design allows the participation of other interest groups that, although they participate in some way in the value chain of health services, are generally excluded from explicit decision-making processes.
The decision-making experiences in oncology based on an MCDA approach provide an input to identify the aspects considered valuable by the different stakeholders. These inputs favor discussion that could eventually contribute to reorienting the goals of care for these patients. In this sense, this study aims for three purposes: a) review the published MCDAs in oncology to identify the criteria considered valuable by the stakeholders, b) evaluate the adherence to best practice recommendations, and c) Propose a value tree, based on the findings of the systematic review, reflecting the most important criteria for the different stakeholders.

MATERIAL and METHODS
A review protocol was developed. However, it was not registered in PROSPERO because it is not currently accepting systematic literature reviews.
This systematic review consisted of the following phases: identification, screening, eligibility assessment, assessment of adherence to best practices, and extraction of the information. The identification was conducted in PUBMED, EMBASE, EBSCO, SCIENCE DIRECT, SCOPUS, LILACS, and Web of Science from February to March 2021. The search included records from January 1st, 1990, to February 28th, 2021, and filters were not used. Because of the lack of a checklist to evaluate the articles' quality, only studies published in journals were included to control the quality of the publication. The strategy and search terms were the following: (῾multi-criteria decision analysis῾ OR ῾MCDA῾ OR ῾multi-criteria decision analysis῾ OR ῾multi-attribute decision analysis῾) AND (῾cancer῾ OR ῾neoplasm῾ OR ῾solid tumor῾ OR ῾lung cancer῾ OR ῾breast cancer῾ OR ῾stomach cancer῾ OR ῾gastric cancer῾ OR ῾prostate cancer῾ OR ῾colorectal cancer῾ OR ῾colon cancer῾ OR ῾cervical cancer῾). The duplicated articles were removed.
Subsequently, the identified articles were screened through titles and abstracts. The records identified as books, conferences, or posters were excluded. In addition, the abstracts that did not apply some MCDA approach or did not evaluate healthcare interventions (e.g., ecology, financial issues) were not considered. Full texts were retrieved of the remaining records. The articles were included if they evaluated therapeutic or rehabilitation alternatives in oncology using an MCDA approach. Additionally, they had to report the methods in detail and be published in English or Spanish.
Methodological articles that evaluated concomitantly other chronic non-communicable diseases (e.g., heart failure, asthma), hypothetical cases, or nontherapeutic interventions (e.g., diagnostic test) were excluded. Finally, those studies that did not perform all stages of an MCDA process (e.g., studies that only performed the identification and weighting of criteria but omitted to score) were also excluded. Studies that met the inclusion criteria were reviewed in depth. Figure 1 shows the flow diagram.
The report "Multiple Criteria Decision Analysis for Health Care Decision Making-Emerging Good Practices" issued by the International Society of Pharmaco-economics and Outcomes Research was used to evaluate adherence to best practices in MCDA (22).
Each article was assigned to two reviewers. The reviewers extracted the information from the included articles independently, which was organized in separate spreadsheets. The collected information included: author, year of publication, country, membership in the Organization for Economic Cooperation Development membership (OECD), cancer site, type and the proportion of participants (e.g., physicians, patients, payers), type of interventions (i.e., medication, surgical procedure, decision system support), problem type (i.e., choice, ranking, sorting), set of alternatives (i.e., stable or incremental), the definition of each criterion, structure of value tree (i.e., flat or hierarchical), methods for weighting criteria (e.g., swing method, direct weighting), strategy for obtaining information (e.g., meeting, workshop) and MCDA modelling approach (e.g., Analytical Hierarchic Process, PROMETHEE).
Subsequently, the data gathered was reviewed jointly. All disagreements were resolved through consensus by the evaluators. If consensus could not be reached, a referee decided on the discrepancy.
Two investigators conducted the identification phase. After that, all investigators took part in the later phases. During these phases, blinding was ensured using the Rayyan platform (23). Finally, the information was synthesized using descriptive statistics.
Aggregation of some types of cancer was made since many of them belonged to the same origin. The hematological group included acute lymphoblastic leukemia, classic Hodgkin lymphoma, chronic lymphocytic leukemia, acute myeloid leukemia, multiple myeloma, adult acute lymphoblastic leukemia, follicular lymphoma, and lymphoma. The colorectal group included metastatic colorectal cancer and rectal cancer. The other types of cancer were not grouped and kept their original name.
Standardization of names for criteria and clusters (i.e., aggregation of similar criteria) was performed. For the standardization, the names of the criteria and clusters proposed by Tanios et al. were selected.
Then, after reviewing the name and definition of criterion in the MCDA, the homologous criterion in the Tanios et al. proposal was identified. In this way, an analysis by categories could be guaranteed, avoiding the large number of criteria identified (24,25).
Finally, after reviewing all the articles, showing the high variability, and identifying the need to homogenize criteria and clusters, it was decided to propose a new value tree to standardize the criteria for decision-making applied to oncology.
This new value tree was made from the results of this systematic review, taking the criteria and clusters we consider most relevant for all stakeholders and adjusted to the field of oncology.

Studies characteristics
Thirteen articles were included in the analysis. The main characteristics of these are presented in Table 1.
Three articles (23%) were written in Canada, two (15%) in the Netherlands, the remaining eight articles were written in other countries. Thus, most of the studies (84.6%) were carried out in countries that belong to the OECD.
Colon and breast cancer were the most frequently evaluated cancers (n=5 articles, 38.4%), followed by hematological tumors (n=3 articles, 23%). Most articles (n=10, 76.9%) evaluated only one type of cancer. The remaining articles (n=3; 23.1%) evaluated two or more tumors. Thus, a single article evaluated 17 different tumor types.
Physicians and patients were the most representative participants in most studies. Physicians were the majority stakeholders in six articles (46.1%), followed by patients (n=5 articles, 38.5%), academics (n=1, 7.7%) and other participants (n=1, 7.7%). The other participants included payers, insurers, pharmacists, or health economists. In terms of percentage representativeness, physicians accounted for 100% of participants in three articles (23.1%), in eight articles (61.5%), their percentage ranged from 8 to 57%, and in two articles, they did not participate. On the other hand, patients represented 100% in one article (7.6%) and did not participate in three articles (23%); in the remaining articles (n=9, 69.2%), their percentage of representation ranged from 14 to 55%.
In terms of participation methodology, the survey followed by meeting (n=3, 23%), workshop alone (n=3, 23%), and meeting alone (n=3, 23%) were the most used. The survey alone was used in two studies (15.4%) and focus group in one study (7.7%). In one study, the description of the methodology used was not clear.

Quality of the included studies
The articles achieved an overall adherence rate to the recommendations of 77.3%. Four articles (31%) met all good practice recommendations. The remaining articles showed adherence percentages ranging from 53 to 94%. All articles achieved full adherence with the "Reporting and examination of finding" step. However, the step "Calculating aggregate scores" had the lowest percentage of adherence to recommendations (54%). The step "Selecting and structuring criteria", one of the main aspects reflecting stakeholder preferences, achieved an adherence rate of 64%. The absence in the report of the value tree was the factor with the lowest adherence in this step (46%).
The details of the methodological quality of each study included are found in Table S2 of supplemental material.
On the other hand, the most relevant criterion for the patients was "Severity of disease" (34.4%). Other significant criteria were "Improvement of clinical efficacy and effectiveness" (24.1%), "Improvement of perceived health status" (13,8%), "Improvement of safety and tolerability" (13.8%), and "Quality and uncertainty of the evidence" (6.9%). For patients, the economic impact is the least relevant item.
Contrary to the two predominant groups, the most relevant criterion for the academic was "Innovativeness of intervention".
The importance of other criteria for all stakeholders is shown in Figure 2.
Ten articles (77%) used clusters to aggregate related criteria. As a result, ten identified clusters aggregated 165 different criteria. The remaining criteria were not aggregated within any cluster. Identified clusters and their corresponding criteria are shown in Table 2.
The analysis by stakeholder group shows that for physicians, patients, and other stakeholders (e.g., health economists, insurers), the most frequently valued cluster was "Intervention outcomes and benefits." In contrast, academics and regulators most frequently valued the cluster "Overall context.". The clusters and the proportion of valuation by each stakeholder group are shown in the supplementary material.
The definitions for each criterion varied widely. The most significant variability in definitions occurred with the criterion "Improvement clinical efficacy and effectiveness" with 38 definitions (19.5%) followed by "Severity of disease" with 21 definitions (10.8%) and "Improvement of safety & tolerability" with 16 definitions (8.2%).
The main definitions for the criteria "Improvement clinical efficacy and effectiveness" included overall survival, comparisons of effectiveness measures between alternatives, and suitability of tumor (i.e., resectability of the tumor). On the other hand, the criterion "Severity of disease" showed definitions such as clinical characteristics of the tumor (e.g., clinical stages, presence of mutations or biomarkers, involvement of surrounding tissues, presence of regional or distant metastases) and the patient's functional status at the time of the procedure. Finally, the criterion "Improvement of safety & tolerability" used definitions related to the severity of adverse effects, probability of complications, and the comparison between alternative safety indicators.
Criteria related to health services management also showed differences in definitions. For example, the definition of "Direct impact of the intervention on healthcare costs" was accepted as direct intervention costs or comparative costs with another intervention. The criterion "Organizational requirement" included locations, workflows in the operating room, capacities, ease of obtaining resources (e.g., disinfection equipment). The criterion "Innovativeness of intervention" accepted definitions such as the phase of the controlled clinical trial and the level in the Anatomical, Therapeutically and Chemical classification. Table S1 in the supplementary material shows the definitions for the different criteria used in the articles.
However, if the alteration is only in the colon, the criterion "Improvement of clinical efficacy and effectiveness" (8/10 criteria; 80%) comes first. This same criterion was the most important for gastric cancer (9/25, 26%).
The inclusion of criteria also depends on the number of cancers evaluated in the same study. For example, when a single type of cancer is evaluated in a single assessment, the criteria "Improvement clinical efficacy and effectiveness" (28/103 criteria; 27.2%), "Improvement of safety & tolerability" (15/103 criteria; 14.6%), and "Innovativeness of intervention" (11/103 criteria, 10.7%) were the most frequently selected. On the other hand, in case several cancers are evaluated, the criteria "Severity of disease" (14/62 criteria; 22.3%), "Direct impact of the intervention on healthcare costs" (9/62 criteria; 15.5%), and "Quality/Uncertainty of evidence" (6/62 criteria; 9.7%) were most frequently selected.

DISCUSSION
This study aimed to review the published MCDAs in oncology, assess their quality, and propose a value tree in this field using those objectives, clusters, criteria, and sub-criteria identified. MCDAs are considered a set of structured techniques to measure the concept of "value" in healthcare (26). This measurement is achieved by identifying, weighing, and scoring criteria for selecting, sort or rank alternatives and competing courses of action. The selection of the criteria attempts to elicit the decision-maker's preferences, which reflect what is considered valuable to them (27).
Despite the usefulness of such studies, their limited use in oncology is noteworthy. Although these techniques were designed and implemented since the 1960s, the publication of studies in oncology is recent. Most of them were published since 2015, which is up to twenty-five years after the start date of our search. Furthermore, there is no upward trend in published articles despite the increase in cancer cases worldwide (28,29). This trend contrasts with the need to include structured tools for making complex decisions due to human reasoning limitations and the necessity for incorporating values (30). This trend also contrasts with the increasing need to adopt shared decision-making approaches that involve other stakeholders (31). A health problem as complex as cancer demands adopting tools adaptable to the multiplicity of situations in clinical practice.
Versatility is one of the characteristics of this type of analysis (32). Our review showed that the same health problem could be approached from different perspectives. For example, the articles included in our review were published in journals belonging to different academic branches (v.g., oncology, informatics, health economics, management sciences), evidencing the multidisciplinary nature of decision making in health (9)(10)(11)(12)(13). In the same way, different types of cancer and interventions were evaluated. Finally, various MCDA approaches (e.g., hierarchical analytical process, MACBETH) were implemented to weight criteria, score the performance of alternatives on criteria, aggregate performance, and rank courses of action.
Even though several MCDA approaches were used in the included studies, the weighted sum models and the hierarchical analytical process were the most common. These two types belong to the value-measurement approach, which is the most frequently used in health. (33) One of the reasons for their widespread use in health is the ease of calculating the aggregation function and their transparency-however, one of the significant limitations is the presence of compensation. (34,35) Compensation arises when an alternative performs worse on one criterion and is better on another than another alternative. Thus, the aggregate value of the first alternative is better than the second alternative, even if it performs worse on one criterion. (34) Although most decisions admit these trade-offs, there are circumstances where ethical issues do not accept these compensations.
Unsurprisingly, the physician was the most representative group. Surprisingly, a significant proportion of the studies did not include the patients. This relatively poor participation has been described in other articles. (36)(37)(38) Other stakeholders such as pharmacists, the pharmaceutical industry, payers, and insurers did not participate in most published articles.
Although it could be considered that the participation of other agents could distort the nature of the doctor-patient relationship by introducing interests in a process in which the patient usually assumes the consequences. However, as Daniels exposes, health care should be the responsibility of social sciences, public policies, legislators, administrators, doctors, and all participating in developing a health care system health (39). In addition, the participation and interaction of different stakeholders are fundamental to agreeing on what is accepted as "health value" in a particular society at a particular historical moment (40).
Our review reveals a moderate adherence to good practice recommendations for these articles. Adherence was mainly achieved in describing the problem, weighing the criteria, scoring the alternatives, and reporting the findings. However, crucial steps such as reporting the methods to derive criteria, representing the criteria in a value tree, reporting the performance matrix, and a clear description of the aggregation function used presented limitations. This relatively poor adherence has been reported in previous studies (41,42). This poor adherence to good methodological practices has serious repercussions because the legitimacy of the decisions obtained is mainly based on the methodological rigorousness of this type of technique.
Several reasons could explain the poor adherence to good practice recommendations, although two are important-first, the absence of health-specific MCDA methodological recommendations. In addition, health workers are often unfamiliar with MCDA concepts, methods, and tools. Second, there are no comprehensible methods in health MCDA on combining individual judgments in face-to-face meetings or survey collection. Oliveira et al. have developed a comprehensive analysis of the factors that explain the poor methodological quality in many of the published MCDA in health (41).

A proposal of value tree in oncology
The concept of "value" in health has changed significantly. One definition of "value" adopts an efficiency approach (i.e., a ratio between quality and costs). From this approach, the patient perspective is minimal. Porter adopts a broader definition "health outcomes achieved which matter to patients relative to the cost of achieving those outcomes." (43). Other authors such as Lawskoski give particular importance to the doctor-patient relationship in the conception of "value" in health (44). Definitively, this concept is much broader and more complex than a simple ratio of inputs and outputs of a production function.
To operate the concept of value, Angelis and other authors have conceived it as a function of parameters.

Value=f(a_ij,b_ij,c_ij,d_ij,…n_ij)
Where a, b, c, d,..n denote the different parameters of interest, i denotes the medical technology's partial value regarding a particular parameter, and j denotes the relative importance weight of the same parameter based on decision-maker or other stakeholder views (45)(46)(47).
The selection of these parameters (i.e., criteria for our case) is the first step in constructing a general value function in oncology. Weighting criteria and scoring alternatives are specific for each alternative y determined by the stakeholders. Although the criteria can be defined according to the alternative or condition evaluated, a general criteria structure helps to indicate how the different actors in the health system should direct their efforts. Moreover, a general criteria structure might facilitate comparison between alternatives in several types of technologies.
These attributes or criteria can be organized in a value tree. A value tree is a graphical representation of the decision maker's objectives, criteria, and sub-criteria. Occasionally it can depict the performance of different alternatives for each criterion. These tools are helpful for explicitly representing the decision-maker's values (48). Having reviewed the literature published to date, we propose a value tree depicted in Figure 3.
Our tree proposal has four levels of complexity: At the first level is the objective to generate value for society during the care of citizens affected by various oncological diseases. To achieve that final objective, we have considered nine clusters which are in the second level. These clusters were chosen because they represent the most significant and essential domains for the different stakeholders. Four clusters (Intervention outcomes and benefits, disease impact, priorities, and quality of evidence) were significant for all stakeholders but especially for patients and physicians. The other five clusters (economic impact, implementation complexity, overall context, types of health services, and environmental impact) were of particular interest to all administrative staff supporting patient-physician interaction.
The third and fourth levels are made up of criteria and subcriteria. A technology in oncology is considered valuable when it is effective, and the effectiveness is based on evidence-furthermore, this technology aims to cure and improve people's quality of life.
Moreover, the technology should be culturally accepted and easy to access and implement for covering a significant number of citizens considering individual factors such as age, family environment, or comorbidities. Regarding administrative management, the technologies should be oriented to prevention, have a lower budgetary impact for providers than existing ones, have minimal requirements for operating, and negligible environmental impact. Finally, from a health system level, technologies add value to the extent that they satisfy the different stakeholders' demands, are innovative and financially sustainable for the health system.
Our study has several limitations. First, even though the proposed value tree synthesizes the most relevant clusters, criteria, and sub-criteria, its construction is based on articles with a moderate adherence to best practice recommendations. This limitation could generate biases in the structure of the tree. On the other hand, most of the studies were conducted in high-income or upper-middle-income OECD countries. For this reason, our proposal does not necessarily reflect the health values of middle-or low-income countries. Our study also did not consider the particularities of the health system where the studies were conducted. Another limitation is that the evaluation of the quality of the studies was carried out using a checklist that requires further validation for its generalized use. Nevertheless, this checklist resulted from rigorous and structured work by an ISPOR Task Force with an orientation toward MCDA in health. Finally, the significant heterogeneity in criteria and cluster names in the published studies made it necessary to standardize the terminology. It is likely that in standardizing the names, some particularities of the study were lost. However, this standardization was performed using a list of internationally validated criteria, which in our opinion, is robust.

CONCLUSION
The increasing incorporation of a "value-based health services" paradigm in oncology requires the use of structured methodologies to identify what is considered valuable to stakeholders. The MCDAs have been consolidated as support for identifying these values, weighing them, and evaluating different courses of action.
Despite the increase in publications of MCDA in health, the number of these studies in oncology is scarce. In addition, several of the published articles present moderate or poor adherence to some recommendations in reporting this type of study.
Finally, we propose a value tree showing the clusters, criteria, and sub criteria recognized as valuable in oncology by the different interest groups.
Author Contributions: The original idea for the project came from CAGC. The review was led and coordinated by CAGC and MPRM. The protocol was written by CAGC, MPRM, GISM, NPS. Subsequently, the protocol was corrected by CAGC and EPR. All authors participated in the search, screening, and selection of articles. In addition, all participated in data collection and analysis. CAGC, MPRM and EPR participated in writing the manuscript. All authors participated in the revision of the article.
Financial & competing interest's disclosure: The authors have no relevant affiliations or financial involvement with any organisation or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.