{'fraction_non_alphanumeric': 0.033194858128233265, 'fraction_numerical': 0.017244082144536762, 'mean_word_length': 4.853865565496673, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Value assessment in oncology drugs: funding of drugs for metastatic breast cancer in Canada

Background Life expectancy for women with metastatic breast cancer has improved since the early 2000s, in part because of the introduction of novel therapies, including chemotherapy, hormonal therapy, and targeted agents. However, those treatments can come at a cost for the patient (shortand long-term toxicities from treatment) and at a financial cost for the health care system. Given the increase in the number of costly anticancer agents being introduced into the clinical setting, the American Society of Clinical Oncology (asco) and the European Society for Medical Oncology (esmo) have developed a system to quantify the value of new cancer treatments in terms of benefit, toxicities, and costs. Methods In our value-assessment analysis, we included drugs that were funded in Canada between 2012 and 2017 for metastatic breast cancer. We reviewed the clinical benefit of those agents (survival, progression, quality of life), their costs, their value according to the asco and esmo value frameworks, and their assessments from the panCanadian Oncology Drug Review [pcodr (in Canada, except Quebec)] and the Institut national d’excellence en santé et en services sociaux [iness (in Quebec)]. Results Drugs funded in Canada showed variation in their asco net health benefit scores and esmo magnitude of clinical benefit scores, but all had a cost-effectiveness ratio greater than $100,000 per quality-adjusted life–year. The strength and magnitude of the clinical benefit (for example, overall survival benefit vs. progression-free survival benefit) was not necessarily associated with a higher value score. Conclusions Although great progress has been made in developing value frameworks, use of those frameworks has to be refined to help patients and health care providers make informed decisions about the benefit of novel cancer therapies and to help policymakers make decisions about the societal benefit of funding those therapies.


BACKGROUND
Survival for patients with metastatic breast cancer (mbca) has improved over the years, at least in part because of the introduction of new cancer therapies 1 . In a recent cohort study of patients starting therapy for metastatic disease between 2000 and 2008, median overall survival (os) was 55.5 months overall, but varied according to tumour biology 2 : median survival was not reached for the luminal-her2 subtypes (estrogen or progesterone receptor-positive, or both, and human epidermal growth factor receptor 2-positive); it was 59.9 months for luminal tumours (estrogen or progesterone receptor-positive, or both, and her2-negative), 49.9 months for the her2-enriched subtypes (estrogen and progesterone receptor-negative, her2-positive), and 18.6 months for triple-negative cancers 2 . In her2-positive mbca, os has been extended by about 4.5 years because of the introduction of her2-targeted therapies 3 .
Nevertheless, improved cancer therapies for individuals with mbca come with a cost. From the patient's perspective, the toxicities associated with cancer treatment, both short-term (diarrhea and fatigue, among others) and long-term (heart failure, for instance), can negatively affect quality of life (qol). In addition, the cost of the drug itself is borne by the health care system and the patient (if not funded or if a co-pay is required), and patients incur out-of-pocket expenses to receive their treatment (for example, parking). Costs are also incurred if patients or their caregivers have to stop working or to take days off work. A survey of the needs of 1577 women with mbca found that more than half the women had to change employment and that the change was associated with a decline in their income 4 . For caregivers, there is a cost for accompanying patients to their appointments. For society at large, there are costs associated with treatments and their complications and with loss of productivity by patients or their caregivers. All of those costs have to be weighed against the benefit that can be derived from treatments so that the value of each treatment option can be estimated and compared. "Value" is defined as a measure of outcomes achieved per level of monetary expenditure 5 . Outcomes can be measured in terms of os, progression-free survival (pfs), treatment toxicities, qol, and capacity to work.
To address the effect of cancer treatments on patients, traditional endpoints such as pfs or os and adverse events are systematically collected in clinical trials; in contrast, patient-reported outcomes and qol might not be collected. Research has demonstrated that qol data supplement systematically collected serious adverse events 6 . For example, a cooperative group found that, in lung cancer clinical trials, toxicities were detected earlier when patient-reported outcomes were collected than when the Common Terminology Criteria for Adverse Events was applied, and that the two methods showed a moderate correlation 6 .
For each new cancer drug, several factors have to be considered before treatment is offered to patients. From a medical point of view, a determination of whether the clinical benefit of the drug outweighs the potential toxicities associated with treatment has to be made. From the patient's point of view, an evaluation of whether treatment-associated toxicity significantly affects qol or duration of life is important. From the societal perspective, a definition of whether the drug is cost-effective is required. In Canada [except in the province of Quebec, where the evaluation role is taken by the Institut national d'excellence en santé et en services sociaux (iness)], the pan-Canadian Oncology Drug Review (pcodr) evaluates new drugs to guide cancer-drug funding decisions. The pcodr framework is based on overall clinical benefit, patient values, cost-effectiveness, and feasibility of adoption into health systems 7 . The American Society of Clinical Oncology (asco) Value in Cancer Care Task Force developed a framework for comparing relative clinical benefit, toxicity, and cost of treatment in oncology 8,9 . This asco initiative is based, at the clinical level, on a standardized approach to help "physicians and patients [assess] the value of a new drug treatment for cancer as compared with one or several prevailing standards of care" and, at the societal level, to assess the proposition that "the cost of a given intervention should bear a relationship to the beneficial impact it has on the patient who receive that treatment" 8 . The European Society for Medical Oncology (esmo) has undertaken a similar process called the Magnitude of Clinical Benefit Scale (mcbs) 10 . Their rationale was that no standard tool was available to grade the benefit of an oncology drug, and so the true benefit might be overestimated. Their intention is to apply the mcbs to all new drugs approved by the European Medicines Agency in the hope that drugs with the highest benefit score could be implemented more rapidly 10 .
In the present study, we selected drugs that were funded in Canada during 2012-2017 for the treatment of mbca, and we used the asco and mcbs frameworks to look at their value and the pcodr and iness evaluations to consider their cost-effectiveness.

Drugs Approved and Funded in Canada for mBCa
In Canada, during the period of interest, these drugs were approved and funded for mbca: lapatinib, pertuzumab, and trastuzumab emtansine for her2 (human epidermal growth factor receptor 2)-positive mbca; eribulin; and everolimus for hormone receptor-positive breast cancer. Palbociclib, which had received a conditional approval by pcodr (if the cost-effectiveness were to be improved to an acceptable level), but which had not yet been funded was also included in the analysis.

Data for the Assessments of Value and Cost
We used the pivotal trial on which each drug approval was based to determine the clinical benefit (pfs and os, when available) and effect on qol. If qol was not included in the pivotal trial or was not reported, we obtained data about qol from another trial using the same drug so as to have a sense of the effect on qol. We also obtained cost-effectiveness results for each drug from pcodr and iness. We then calculated the asco net health benefit score 8,9 and the esmo mcbs 10 for each cancer drug.
Obtaining the asco net health benefit score is a 6-step process ( Table i). The first two steps determine a regimen's clinical benefit and toxicity, and bonus points are awarded in the third step if the regimen shows either or both of improvement in the palliation of symptoms or in the treatment-free interval compared with control subjects. The clinical benefit, toxicity, and bonus points are then combined to generate a net health benefit score (step 4), which can then be weighed against the direct cost of the treatment (step 5) and integrated into an overall summary assessment of the treatment's value (step 6). We used data from the pivotal trial only (no indirect comparisons were made-for example, for qol-if relevant data were not obtained in the pivotal trial).
The scoring grid for the esmo mcbs (Table ii) varies according to the primary endpoint (os vs. pfs) and the median os (more or less than 1 year) or pfs (more or less than 6 months) in the standard-treatment arm. Points are added or subtracted depending on qol and toxicity. In the noncurative setting, the score varies from 1 to 5 (when os is the primary endpoint), with scores of 4 and 5 being considered substantial improvements 10 .
To estimate the costs of the drugs, we used an average body surface area of 1.73 m 2 and a weight of 70 kg 7 . For oral drugs, we used the costs from the Liste des médicaments published by the Régie de l'assurance maladie du Québec as of April 2017 and from a drug wholesale company in the province of Quebec.
All data are presented descriptively. Step Score 1 Clinical benefit score (A) Is the hazard ratio (HR) for death reported? HR score (death): _ YES Report, and proceed to 1(F).
Assign HR score for death by subtracting the HR from 1 and then multiplying the results by 100.
NO Proceed to 1(B) (B) Is the median overall survival (OS) reported? OS score: _ YES Report, and proceed to 1(F).
Assign OS score by calculating the percentage (that is, fractional) difference in median OS between the two regimens and then multiplying the result by 100.
(C) Is the HR for disease progression reported? HR score (progression): _ YES Report, and proceed to 1(F).
Assign HR score for disease progression by subtracting the HR from 1, multiplying the result by 100, and then multiplying that number by 0.8.

NO Proceed to 1(D).
(D) Is the median progression-free survival (PFS) reported? PFS score: _ YES Report, and proceed to 1(F).
Assign PFS score by calculating the percentage (that is, fractional) difference in median PFS between the two regimens, multiplying the results by 100, and then multiplying that number by 0.8.

NO Proceed to 1(E).
(E) Is the response rate (RR) reported? RR score: _ YES Report, and proceed to 1(F).
Assign RR score by adding the complete response (CR) and partial response (PR) rates, multiplying by 100, and then multiplying that number by 0.7.
(F) Insert the HR score (death), OS score, HR score (progression), PFS score, or RR score as already determined.

Anti-HER2 Therapies
Lapatinib, an oral tyrosine kinase inhibitor, was studied in her2-positive mbca patients who progressed after a taxane, an anthracycline, and trastuzumab. The EGF100151 trial randomized such patients to either capecitabine alone or capecitabine-lapatinib [11][12][13][14] . The primary outcome was achieved, with an improvement in time to progression (4.4 months vs. 8.4 months). Of the patients participating in that trial, 60% in the combination arm compared with 39% in the monotherapy arm experienced any-grade diarrhea; the percentages for rash were 27% and 15% respectively. No difference in qol was observed. According to the asco value framework, the net benefit was 50.4, with a monthly cost of $4,367 for the combination of lapatinib with the original form of capecitabine and $842 for (original) capecitabine alone. The esmo mcbs was 3-less than the score of 4 that is considered a substantial improvement. Pertuzumab in combination with trastuzumab was approved based on the results of the cleopatra trial 15,16 . That randomized trial of trastuzumab-docetaxel plus either placebo or pertuzumab in the first line in mbca found pfs and os benefits, with the median survival improving to 56.6 months from 40.8 months. Even though toxicities such as diarrhea, headache, and fatigue were more common with the addition of pertuzumab, the qol analysis found no significant difference between the treatment arms 17 . In a post hoc analysis, the median time to deterioration in breast symptoms was 26.7 weeks in the pertuzumab arm and 18.3 weeks in the placebo arm [hazard ratio (hr): 0.77; p = 0.0061]. The asco net health benefit score was 48.4, and the monthly cost was $10,932 for the combination of pertuzumab with trastuzumab when given with docetaxel. The esmo mcbs score was 4.
The latest anti-her2 therapy to be approved was trastuzumab emtansine, which was found in the emilia trial 18 to be superior to capecitabine-lapatinib in terms

Item Score
If survival is the primary endpoint ...    of pfs and os, and to be associated with an improvement in median survival to 30.9 months from 25.1 months. In addition, less toxicity (mainly diarrhea and palmarplantar erythrodysesthesia) was observed with trastuzumab emtansine than with capecitabine-lapatinib. Those differences were associated with a longer time to worsening of symptoms as measured by the Functional Assessment of Cancer Therapy-Breast (FACIT.org, Elmhurst, IL, U.S.A.) and the Trial Outcome Index (physical/ functional/breast) 19 . The asco net health benefit score was 63.6 for an average monthly cost of $7,488, and the esmo mcbs score was 5, which is the highest possible score, given the improved os and better qol. Cross-trial comparisons are difficult, given that these drugs were not administered in the same population of patients with mbca (for example, first-line vs. second-line therapy) and had different comparators. In the second line, trastuzumab emtansine and lapatinib have similar cost-effectiveness ratios, although trastuzumab emtansine had the highest asco net health benefit score and the highest esmo mcbs.

Hormone Receptor-Positive, HER2-Negative mBCa
Everolimus was approved in combination with exemestane for hormone receptor-positive, her2-negative mbca, based on improved pfs in the bolero-2 trial 20 . The median os was numerically longer in the combination arm (31.0 months vs. 26.6 months), but the difference was not statistically significant (hr: 0.89; p = 0.14) 21 . Although a numerically higher proportion of patients experienced a deterioration of more than 5% from baseline in the combination arm (52% vs. 47% in the placebo arm), the median time to deterioration in health-related qol (global health status) was 8.3 months in the exemestane-everolimus arm and 5.8 months in the exemestane-placebo arm (hr: 0.74; p = 0.0084) 22 . For everolimus, the asco net health benefit score was 46.8, with a monthly cost of $5,735 when everolimus was combined with the original form of exemestane and $155 when exemestane was given alone. The esmo mcbs was 3.
Palbociclib is a member of the cyclin-dependent kinase 4/6 inhibitor family and, at the time of writing, was the only drug in this class to have received Health Canada approval (no funding decision has yet been made, however). The paloma-2 randomized trial in the first-line metastatic setting compared letrozole-placebo with letrozole-palbociclib. The os data are not yet mature, but pfs, the primary study endpoint, was positive, with a median pfs of 24.8 months in the letrozole-palbociclib arm compared with 14.5 months in the letrozole-placebo arm (hr: 0.58; p<0.001) 23 . Patient-reported outcomes for paloma-2 have not yet been published, but outcomes in paloma-3, a trial of fulvestrant plus placebo or palbociclib, have been 24,25 . The asco net health benefit score was 44.8, with a monthly cost of $7,128 for the combination with the original form of letrozole and $164 for letrozole alone. The esmo mcbs was 3.
Palbociclib and everolimus have similar values when either the asco net health benefit score or the esmo mcbs is used. However, the absolute difference in pfs shows large variation, although the hr is similar. The fact that the data come from different settings (first-line palbociclib vs. second-line everolimus) is not ta ken into account.
Despite a large absolute benefit in pfs for palbociclib and endocrine therapy, the cost-effectiveness ratio falls into the range $390,200-$501,799 per quality-adjusted lifeyear (qaly) 29 .

Chemotherapy
The only chemotherapy drug that was approved during the period of interest is eribulin. When compared with a treatment of the physician's choice, eribulin was shown to improve os in the phase iii randomized embrace trial 26 . The trial had no qol endpoint, but another phase iii trial comparing eribulin with capecitabine found no difference in global qol over time 27,28 . Eribulin was associated with an asco net health benefit score of 18.3 and a monthly cost of $3,038. The esmo mcbs was 2. Although this drug was associated with a survival benefit in heavily pretreated patients with mbca, it had the lowest value framework scores.

DISCUSSION
Two major oncology associations, asco and esmo, have each worked on a value framework scoring system that takes into consideration clinical benefit, toxicities, and the qol associated with new cancer therapies. Furthermore, the asco framework adds the notion of cost. A study comparing the asco net health benefit score with the esmo mcbs found only weak-to-moderate correlation 30 . Another study observed a negative correlation between the asco net health benefit score and incremental cost 31 .
Although cost-effectiveness is important for the health care system, it is a vague term for patients, and the cost of drug, and therefore its impact, can vary according to the patient's insurance coverage and income. From the patient's perspective, a cost per month, as presented by asco, is easier to understand than a cost-effectiveness ratio. The drug evaluation mechanism in Canada also considers all those elements and decides in the end to recommend funding (or not) based on the cost-effectiveness ratio, provided that the drug has clinical benefit. Given the limited financial resources of the health care system, a report of clinical benefit is not enough to recommend funding a drug. In the province of Quebec, the iness report of a drug evaluation also considers the "significance" of the budget impact if the new drug is approved. For example, the additional cost of providing palbociclib for patients with estrogen receptor-positive mbca in Quebec for 1 year was estimated to be $21,235,059 29 , which corresponds to 307,755 hours of home-care nursing, or 345 long-term beds, or 312 palliative care beds, or 91,530 days of emergency care for 1 year.
At the societal level, all drugs for the treatment of patients with mbca reviewed in the present analysis were shown to have a clinical benefit for women with mbca, but with a cost-effectiveness ratio exceeding $100,000 per qaly. At the patient level, the clinical benefit varied: some drugs were associated with a survival benefit (for example, pertuzumab, trastuzumab emtansine, and eribulin), and others were not (lapatinib, everolimus). Value is also patientdependent: for some, duration of survival is most important; for others, qol is the primary objective. Patient goals and expectations must be taken into account. Fortunately, most trials report pfs, response rates, os, and toxicities.
Quality of life is a secondary endpoint in many trials, but not all. The most recent guidelines from asco concerning endocrine therapy for hormone receptor-positive mbca found only four trials in which qol was measured 32 . That observation emphasizes the need for evaluation and reporting about the effects of new treatments for a patient's qol. Studies have shown that, compared with no progression, disease progression is associated with a worsening of qol 33,34 . For example, in a prospective Canadian study of qol and utility in 202 patients with hormone receptorpositive, her2-negative mbca, qol was found to be better for patients in a progression-free state than for patients with progressive disease 34 .
Value frameworks have also been developed for other disease areas such as cardiology 35 , and in oncology, the asco and esmo frameworks are just two of the available possibilities. Others include the U.S. National Comprehensive Cancer Network's evidence blocks 36 , the Institute for Clinical and Economic Review's evidence rating matrix 37 , and DrugAbacus 38 . Comparison of those five tools found differences in target stakeholders and outcomes presentations, among other aspects 39 . In an era of "personalized" medicine, we ideally need a "personalized" value-assessment tool, because the benefits and toxicities of treatment can vary greatly between individuals. As suggested by Wong et al. 40 , assessment of the patient's priorities is essential to personalize the value of a treatment that can differ from one patient to another 41 . Also, the evidence used for value inputs should ideally reflect, from either randomized trials or real-world evidence 39 , the heterogeneity of the patient population and the heterogeneity of responses. For example, immunotherapy might produce "long term" survivors in diseases for which such longer survival has not been observed in the past 42 . Clinicians will have to become familiar with the value tool before presenting it to patients. Currently, the most commonly used "value" assessment for cross-discipline comparisons is the incremental cost-effectiveness ratio. For example, in prevention, vaccination against the human papillomavirus in Grade 8 girls is associated with an incremental cost-effectiveness ratio of less than $10,000 per qaly 43 . By contrast, in cardiology, left ventricular assist devices for patients with end-stage heart failure not eligible for transplantation is associated with an incremental cost of $230,692 per qaly 44 .
Our study has limitations. We did not conduct a systematic review, and our analysis was restricted to pivotal trials only. Since the publication of those trials, other trial results have been made available, and they do not always show the same magnitude of benefit found in the original trial 27,45 . In addition, the trials were comparing a new drug with the previous standard. For a patient who wants to know about the benefit between "active treatment" and "palliative care only," no information about the absolute benefit is available, except through the use of cross-trial comparisons 3 .
Second, we used publicly available price lists; we did not have access to the prices that are negotiated between industry and the provincial ministries of health or hospitals.
Third, we encountered some difficulties with the calculation of the toxicity score within the asco value framework. The first problem concerned the categorization of adverse events according to their grades and frequencies, which might not adequately reflect the entire toxicity of a treatment because the calculation does not take into account the actual percentage in each arm. The second problem is that the toxicity score excludes laboratory results, and some abnormal laboratory results (such as hemoglobin) could possibly be associated with symptoms. The third problem was that, as with any trial, calculations are based on results from a selected patient population. When those results are applied in an unselected population, external validity in terms of the clinical benefits and toxicities of the new therapies is not known. The final problem was that, in routine clinical practice, the score cannot be calculated. To help with score calculation, asco is planning for a Webbased application. Some difficulties with the esmo mcbs also arose in categorizing toxicities for patients. Those difficulties are reflected in small differences between our scores and other scores published for the same drugs 31,46 .
Finally, we must acknowledge that other value frameworks exist 39 ; however, we elected to use two from large oncology associations, together with two from the Canadian health care system (pcodr and iness).
In an era of novel cancer therapies, particularly targeted therapies with dual inhibition, formal assessment of the value of those drugs from a patient and a societal perspective will become even more important for providerpatient shared decision-making and for governmental decisions about drug funding. More importantly, a way has to be found to communicate with patients about treatment options (for example, with decision aids 47 ) and to align those options with their values.

CONCLUSIONS
Although great progress has been made in developing value frameworks, the use of those frameworks has to be refined so that patients and health care providers can make informed decisions about the benefit of novel cancer therapies and so that policymakers can decide on the societal benefit of funding those therapies.