{'fraction_non_alphanumeric': 0.02295718794680191, 'fraction_numerical': 0.005611157571016213, 'mean_word_length': 5.44359464627151, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Stakeholder-Engaged Derivation of Patient-Informed Value Elements

Objectives Our objective was to identify patient-informed value elements that can be used to make value assessment more patient centered. Methods Mixed methods were used iteratively to collect and integrate qualitative and quantitative data in a four-stage process: identification (stage 1), prioritization (stage 2), refinement (stage 3), and synthesis (stage 4). Qualitative methods involved one-on-one discussions with 14 patient stakeholders from diverse medical communities representing mental health, osteoporosis, blindness, lupus, eczema, oncology, chronic obstructive pulmonary disease, and hypercholesterolemia. Stakeholders completed guided activities to prioritize elements important to patient healthcare decision making. Responses were summarized descriptively as frequencies and proportions. Results Stakeholders identified 94 value elements in stage 1. Of these, 42 elements remained following the stage 2 prioritization and the stage 3 refinement. During the stage 4 synthesis, the 42 patient-informed value elements comprised the principal set of value elements that were organized by 11 categories: tolerability, disease burden, forecasting, accessibility of care/treatment, healthcare service delivery, cost incurred on the patient, cost incurred on the family, personal well-being, stigma, social well-being, and personal values. The categories fell under five domains: short- and long-term effects of treatment, treatment access, cost, life impact, and social impact. Conclusions In total, 75% of the value elements in the conceptual model were patient derived and distinct from the elements used in existing value frameworks. Recommendations for tailoring, quantifying, and applying the patient-informed value elements in distinct patient communities are provided. This provides a foundation from which future research may test patient-informed value elements in existing value frameworks and economic evaluations. Electronic supplementary material The online version of this article (10.1007/s40271-020-00433-8) contains supplementary material, which is available to authorized users.


Introduction
Value assessment frameworks have emerged as a way to inform a range of audiences about the value of health technologies under review. Existing value frameworks are anchored on value measures related to clinical outcomes and their associated costs [1][2][3]. Recent criticisms are that these elements do not reflect other constructs of value important to patients [4], they are not patient centered, the inputs are not informed by patients [5], and there has been insufficient patient engagement in the process [5,6]. Consequently, current value frameworks are unable to weight the results according to treatment preferences, even though it is known that patients value specific aspects of treatment differently [7].
Several recent reports draw attention to the advancement in novel value measures, where gaps exist, and where more work is needed to make value assessment more patient centered. A focus has been on identifying novel value elements [8,9], such as severity of disease, equity, or value of hope [7,9,10]. The International Society for Pharmaceutical Outcomes Research (ISPOR) Value Assessment Frameworks Special Task Force Report [11] recently emphasized the need for patient perspectives in value assessments [7] and offered a set of recommendations to address gaps in the field, one of which is to test novel elements [6,8,9,12]. This would begin to address the call for meaningful patient input in value assessment [6,[13][14][15].
Some barriers to making value assessments more patient centered do exist. For one, there appears to be no set of empirically derived patient-informed value elements, and this hinders their application in existing value assessment frameworks. To adequately reflect the views of a patient community, we must elicit directly from patients their values, experiences, and needs for outcomes and treatment options. Without meaningfully engaging patients, value assessments may not authentically reflect their perspectives on what is most important in healthcare decisions. Second, guidance on how to quantify patient-informed value elements consistently in value assessment is limited. The Patient-Driven Values in Healthcare Evaluation (PAVE) Center was established at the University of Maryland, USA, with the specific goals of using a stakeholder-engaged and empirically driven process to identify a set of patientinformed value elements, establish quantifiable measures that can be incorporated into existing value frameworks, and test methods for using these measures in economic evaluations. The goals of this paper are to describe our process of identifying a core set of patient-informed value elements with continuous patient-stakeholder engagement and to provide guidance on methodological approaches to quantify and apply the patient-informed value elements.

Design
Qualitative and quantitative methods were used iteratively to elicit patient-informed value elements, defined in this paper as elements that patients consider important when selecting among treatment and healthcare options. Qualitative methods involved one-on-one discussions with patient stakeholders. Quantitative methods included guided activities that were administered via an electronic instrument. A methodological iterative process to integrate qualitative and quantitative data collection and synthesis was facilitated by the four stages described in the following sections. This work was deemed exempt as non-human subjects research from the University of Maryland Baltimore Institutional Review Board.

Patient Stakeholder Involvement
Three patient representative members of the PAVE Stakeholder Advisory Committee (SAC) were involved in all stages of this work. The SAC patient representatives have first-hand experience advocating for the healthcare needs of patients. One patient stakeholder is involved in national policy and advocacy for the rare disease patient community, one delivers patient support to one of the largest behavioral health providers in a mid-Atlantic state, and one represents Hispanic communities.
Individuals from a broader patient stakeholder community were involved in the final phase of the value-element identification. In total, 14 individuals representing eight diverse national patient advocacy groups, which included mental health, osteoporosis, blindness, lupus, eczema, oncology, chronic obstructive pulmonary disease, and hypercholesterolemia, were members of the National Health Council (NHC) Value Workgroup and familiar with value assessment. This workgroup was an established entity preceding this work.

Patient-Informed Value Element Development
A staged qualitative and quantitative methodologic approach engaged patient-stakeholder representatives directly to elicit the elements of value important to the patient community. The three PAVE SAC patient representatives met via telephone with the first author (SDR) three times to identify the issues that are important to patients in making healthcare decisions. It was through these iterative discussions that they derived a list of value elements, which were compared with value elements in the published literature. Representatives from the broader patient community then assessed the importance of the value elements. Four stages were used to arrive at patient-informed value elements subsequently organized into a comprehensive core set.

Stage 1: Identify Existing Elements of Value Important to the Patient Community
An important initial step was to identify any congruence and differences in elements of value defined by patients and those assessed in research. A thorough review of the published literature using PubMed and Embase identified the breadth of elements incorporated into existing value assessments. The search terms to retrieve relevant literature are provided in the electronic supplementary material (ESM; Appendix Table A1). Two research assistants scanned the articles to assess the relevance of the peer-reviewed publications with respect to the elements in existing value frameworks, patient-centered research on healthcare decision making, and value elements in economic evaluations. We considered how researchers and value framework developers defined and measured the elements of value and how this was incorporated into value frameworks or healthcare decision making. The result of this review was a list of existing elements used to assess the value of healthcare interventions.
In the first round of review and discussion, the three SAC patient representatives provided input on elements of value to patients in making healthcare decisions. The goal for this round was to first familiarize everyone with the current landscape for value assessment and elicit the elements that are important to the patient community. Next, they considered the list of elements that emerged from the literature review and commented on whether any elements were missing. During the group discussion, elements were added, some elements were rephrased, and some were marked for deletion. We incorporated this feedback and revised the element list.
In the second round of review and discussion, the SAC patient representatives examined the revised element list in greater depth. The goals of this round were to confirm the changes from the prior discussion, obtain advice on whether elements were missing after the first round, and, if yes, what elements needed to be added, and to remove elements that were not important to patients in making healthcare decisions. Each SAC patient representative reviewed the list independently and made suggestions for additions, deletions, or re-phrasing. During the group discussion, all input was reviewed, and any differences of opinion were resolved through consensus. At the conclusion of the discussion, a master list of patient-informed elements incorporated the collective feedback from the three SAC patient representatives.
The third and final round of review and discussion focused on defining each element and confirming whether further edits were needed. The goal was to capture the meaning of the patient-informed element in the voice of the patient. Again, each of the three SAC patient representatives independently reviewed the list, revised from the previous round, and provided a brief definition of each element given their respective lens. The group discussion was used to refine the meaning and achieve consensus on the definition.

Stage 2: Prioritize Patient-Informed Value Elements
The second stage of this work assessed the validity and completeness of the patient-informed value elements using a wider patient community that represented various medical conditions. The patient-community representatives were recruited from the membership of the NHC Value Workgroup and were invited to participate in a guided activity with two of the authors (SDR, YDH) where they could provide feedback on the patient-informed elements. We developed an online tool (Appendix Table A2 in the ESM) in Qualtrics ® that elicited input on whether the element label and definition required rephrasing to improve clarity, and if so, what would they recommend. Individuals were also asked to rate whether the element was of high, medium, or low importance when making decisions about a healthcare intervention. The one-on-one guided activity was conducted via WebEx ® teleconference where the tool could be screen shared to facilitate the guided activity. For each element, individuals provided their input by answering three questions: Would you rephrase the element label or the definition? (yes/no); If yes, how would you rephrase the element label or the definition? (open-ended); How important to treatment decision making is this value element to the patient community? (high/medium/low). The research team reviewed the feedback, modified the definition phrasing where recommended, and summarized the importance ratings.

Stage 3: Refine Patient-Informed Value Elements
The third stage ensured we did not overlook any elements that are important to patients, confirmed that elements flagged for removal in stage 1 should be removed, and implemented any additional refinements. The three SAC patient representatives independently reviewed the 47 elements flagged for removal after stage 1 and provided input on whether to delete, retain, or combine the element with an existing element, and noting which one. A research assistant consolidated the feedback from the three independent reviews, and elements where two or more patient stakeholders recommended deletion were removed.

Stage 4: Synthesis of Patient-Informed Value Elements
The final stage synthesized the value elements into the core set of patient-informed value elements. First, the elements that reflected the same construct were grouped into categories. Next, the categories were clustered into broader domains. The three SAC patient representatives reviewed the model of thematic domains, categories, and individual elements and provided suggestions for changes. All three SAC patient stakeholders approved the final principal set of value elements.

Identifying Elements Important to Patient Values in Healthcare
Of the 94 elements of value identified in stage 1, there were 60 from the literature and 34 from patient stakeholder input (Fig. 1). The SAC patient representatives flagged 47 elements for deletion. Of the 47 elements retained for the candidate set in the stage 2 prioritization, 26 were derived directly from patient experiences and seven were retained from the literature as being important to patients (Appendix Table A3 in the ESM). An additional five elements from the literature were rephrased and nine elements were modified to be more  Table A3 in the ESM). The definitions of the patient-informed value elements are in Table 1.
After the final round of stage 1 discussions, three of the elements, i.e., benefit-risk tradeoff for patients, benefit-risk tradeoff for the community, and maximum willingness to pay for benefit gained were eliminated, leaving 44 value elements for the stage 2 prioritization. The research team, in consultation with the three SAC patient representatives, determined that these three elements were not independent elements but rather a combination of more than one element. It was not clear what the benefit or risk tradeoff represented, and it was determined that these would be redundant with the existing element explanation of treatment benefits and risks.

Prioritizing Elements Important to Patient
Values in Healthcare Table 2 illustrates the value elements rated as high priority after the stage 2 evaluation. A total of 32 elements were rated high priority by ≥ 50% of the patient stakeholders and only three elements were rated high priority by < 25% of the patient stakeholders. The 12 elements deemed high importance by > 75% of the patient stakeholders were tolerability, side effects, ability to maintain relationships with family, ability to work, impact on depression, affordability, longterm costs, reimbursed care, available treatment, appropriateness of care, provider willing to deliver care, and explanation of the treatment benefits and risks. The three elements rated as of high importance by < 25% of the patient stakeholders were conflict with religious beliefs, fear of rejection by family, and fear of rejection by society. Elements such as social activities, support network and relationships with family and peers, and the physical impact on fatigue, ability to work, impact on anxiety, life expectancy, and the financial impact of treatment were of high importance for 50-75% of patient stakeholders. Notably, none of the patient stakeholders recommended deleting any elements or adding missing elements.

Refining Patient-Informed Value Elements
The research team and the three SAC patient representatives made minor revisions to the value element list after stage 2. The addition of four new elements included physical abilities, embarrassment/self-conscious, autonomy/ dependence, and medication frequency. Physical abilities included getting ready in the morning and ability to exercise, which was unique from ability to work. Stigma became an overarching category that included embarrassment/self-conscious along with rejection by society and family. Autonomy/dependence was a related but distinct element from relocation costs to be closer to family and from long-term effects on the family. Medication frequency was an aspect of tolerability, so that became an element and tolerability the overarching category. Two elements, impact on depression and impact on anxiety, were combined into emotional status. Finally, alternative treatments, pharmacy access to medication, and treatment access in socially neglected populations were absorbed in available treatment and not retained as unique elements. The patient-informed value element development in stages 1 and 2 is in the ESM (Appendix Table A3). This resulted in 42 unique patient-informed value elements. The element refinement in stage 3 is shown in the ESM (Appendix Table A4). The three SAC patient representatives confirmed removal of 26 of 47 elements flagged for removal after stage 1. Of the 21 that remained, all were combined or encompassed within an element that emerged from stage 1 or 2. No new elements were added to the list of 42 patient-informed elements that emerged from stage 2.

Synthesizing a Core Set of Patient-Informed Value Elements
The 42 patient-informed value elements were organized into 11 categories that were subsumed within five thematic domains (Fig. 2). The 11 categories included tolerability, disease burden, forecasting, accessibility of care/treatment, healthcare service delivery, cost incurred on the patient, cost incurred on the family, personal well-being, stigma, social well-being, and personal values. The categories of tolerability, disease burden, and forecasting were within the domain of short-and long-term effects of treatment. The treatment access domain included accessibility of care/treatment and healthcare service delivery. Cost incurred by the patient and cost incurred by the family were in the treatment costs domain. The life impact domain included personal wellbeing and stigma, and the social impact domain included social well-being and social values. The individual patientinformed value elements within each category are listed in Fig. 2.

Recommendations for the Application of Patient-Informed Value Elements
It was not the intent to develop a single metric or a patientreported outcome measure, because not all elements are important to all groups. These elements ideally would be tailored to a specific patient stakeholder group. To implement this in patient-centered value assessment, the research team developed recommendations for the methodological quantification and application of the elements.

Patient-driven value element Definition
Intermediate/surrogate endpoints A treatment endpoint that may correlate with a true endpoint but does not always guarantee the true endpoint will be achieved Tolerability The ability to endure treatment (side effects, dosing, administration burden, etc.) Symptom importance Preference for some treatments over others, depending on the symptoms it can alleviate New therapeutic option New drug option that represents an innovative or breakthrough therapy Side effects The burden that the effects of medication present Life expectancy The degree to which the symptoms of a particular condition limit one's normal/ expected life expectancy Conflict with religious beliefs A treatment, intervention, or anything related to receiving the therapy that presents a conflict with one's religion Age of onset The impact that the age of onset of a health condition plays into the personal benefitrisk assessment in therapeutic decision making.

Cultural barriers
A treatment, intervention, or anything related to receiving the therapy that presents a conflict with one's cultural practices or beliefs Stigma The treatment, intervention, or anything related to receiving the therapy that presents a negative impact in the context of one's society, culture, or beliefs Length of treatment The impact that the duration of treatment/intervention may have on burden in one's daily life Maintain social activities The ability to continue activities in one's social role during the treatment of a disease Support network Family, friends and/or a peer group, or community that lends support and encouragement during treatment Ability to maintain relationships with peers The extent to which the treatment, intervention, or anything related to therapy impedes one's ability to maintain social relationships Ability to maintain relationships with family members The extent to which the treatment, intervention, or anything related to therapy impedes one's ability to maintain family relationships Fatigue The impact of a treatment, intervention, or anything related to therapy on one's physical and/or mental strength Ability to work The treatment, intervention, or anything related to therapy that allows or impedes one's ability to work Impact on depression The effect of the treatment, intervention, and/or anything related to therapy on improving or worsening depression Impact on anxiety The effect of the treatment, intervention, and/or anything related to therapy on improving or worsening anxiety Fear of rejection by family The fear of explaining a treatment, intervention, or anything related to therapy to family due to concern about rejection Fear of rejection by society The fear of explaining a treatment, intervention, or anything related to therapy to people in society due to concern about rejection Available treatment The treatments, interventions, or therapy that are available based on the disease and/or the location of the patient Treatment accessibility in socially neglected populations Individuals with rare diseases or that are otherwise socially neglected populations have fewer treatment options and/or lack access to treatments Impact on education For some diseases, the impact of the treatment on one's education/schooling Impact on career The impact of treatment on one's career Affordability A treatment, intervention, or anything related to therapy that is/is not within one's ability to pay for Cost of treatment-related side effects The cost to an individual and/or society to treat the side effects that arise from the treatment Long-term costs The ongoing costs of treatment and anything related to therapy (i.e., caregiving, etc.) that contribute to financial burden Sibling costs The burden of disease through the lens of siblings, i.e., sacrifices made and opportunities lost to siblings of an individual with a disease that may be pediatric or adult onset Long-term effects on family The impact of a treatment, intervention, or anything related to therapy that can affect the family as a unit, financially or otherwise, over a 10-year time span and more

Recommendation 1: Tailoring Patient-Informed Value Elements to Stakeholder Groups
To tailor the core set of 42 patient-informed value elements to those that are most important to a patient community with a specific medical condition, we recommend that stakeholders be engaged in individual interviews or in focus group discussions. We recommend that individuals are asked to identify the most important elements within each of the five domains. From here, further reduction can be accomplished by identifying the top priorities across domains. Future research may consider literature-based approaches and the US FDA Patient-Focused Drug Development reports for specific medical conditions to identify a starting point for each subgroup. The end goal for use of the value elements-be it clinical practice guidance, benefit plan design, utilization management decisions, or new drug development-should guide this process.

Recommendation 2: Quantifying Patient-Informed Value Elements
Once the core set of patient-informed value elements is tailored to a patient community, these can be quantified depending upon the desired end goal. It is possible to evaluate the rank-order importance in a descriptive assessment of the value elements. The research team used stated-preference methods to quantify preferences using regression models [16][17][18][19]. The resulting beta estimates, or preference weights, inform their relative importance in patient-informed value-based decisions. It is also possible to evaluate the trade-off among different value elements using the preference weights.

Patient-driven value element Definition
Relocation costs to be closer to patient Costs to a family member or the individual undergoing treatment that is related to relocation in order to be closer to family so that caregiving, transport to appointments, clinical care, and care delivery is possible and/or feasible Benefit-risk tradeoff for the community An individual's perspective of the risk versus benefit for the community before accepting and/or paying for a treatment Benefit-risk tradeoff for patients An individual's assessment of whether the benefit of a treatment is worth the potential risk associated with the treatment, or vice versa Maximum tolerable cost for benefit-risk tradeoff The highest amount one is willing to pay for a treatment, intervention, or anything related to therapy Appropriateness of care The treatment chosen is the right intervention or therapy given the individual's needs and preferences System navigation A group/person or tools that can help individuals navigate the system of care more easily Provider relationship and trust The trust one has in the system of care and the healthcare providers that help one make treatment decisions and/or access care Proximity to care location Treatment that is or is not accessible in or near one's geographic locale Reimbursed care The amount of treatment or therapy costs that is covered by insurance or a third-party payer Provider willing to deliver care Having a provider in one's insurance network that is able to deliver or offer the treatment needed Pharmacy access to the medication Access to the prescription medication at a pharmacy within one's community, i.e., treatment that is not restricted to specialty pharmacies or distribution networks Alternative treatments Nonconventional treatments that may/may not be available as alternatives to existing treatments and/or therapy Consistency of care Consistency with respect to the receipt of treatment Care transitions A change to a new or different healthcare facility that impacts access to a treatment Explanation of treatment (risks and benefits) The ability of the healthcare provider to explain to the patient the expectations during the treatment Predictable healthcare needs The variability in a condition and ability to predict one's care and treatment needs over the disease trajectory Inability to plan The ability to plan for one's future, care needs, treatment, interventions, and/or anything related to therapy

Recommendation 3: Applying Patient-Informed Value Elements in Economic Evaluations
Value elements that have been quantified for a particular patient community can ultimately inform components of value assessments. Some patient-informed value elements such as patient costs already exist in economic evaluations that use the societal perspective, whereas others are novel and rarely considered in evaluations, such as sibling costs, stigma from family and society, personal values, and social well-being through support and relations with family and peers. Domains and attributes for health state utility estimation should be compared with the elements ranked highly by the patient community to ascertain whether quality-of-life instruments measure the prioritized value elements. The patient-informed value elements also may be a platform from which to assess the patient centeredness of widely used quality-of-life instruments. If not, alternatives such as stated-preference methods may be explored to quantify patient preferences for health states and quality of life. Our patient-informed value elements can address the patient-centeredness gaps identified in existing value frameworks [5,6]. Examples include weighting elements based on patient preferences, ensuring impact is given to qualitatively assessed elements, and measuring tradeoffs for productivity, illness burden, and innovation [5,6].

Discussion
This set of patient-informed value elements was directed by patient stakeholders throughout the entire process and was grounded in patient experiences that enriched the contextual meaning and relevance of each element. Our SAC patient representatives and the patient communities involved in this work reflected patients across the age range, from caregivers of young children to older individuals living with chronic conditions. In total, 75% of the value elements in the core set were patient derived and distinct from the elements used in existing value frameworks. Notably, many of the value elements highlighted in a recent ISPOR task force report [9], such as fear of contagion, real option value, scientific spillovers, and equity were not endorsed by patient stakeholders.
Of the three SAC patient stakeholders involved in guiding this formative work, one was well-versed in value assessment and familiar with the topics to guide the discussion in their relevance to patients. It is possible that these are more relevant to societal and population perspectives in a situational context than they are to the patient perspective. The ISPOR task force report noted that improving adherence is a common value element but is rarely or inconsistently measured even though patient values for treatment attributes can indirectly affect health through adherence [9]. Our patient-informed elements can build upon this call for clarification and research on adherence-improving factors. The recently updated Institute for Clinical and Economic Review (ICER) 2020-2023 value assessment framework explicitly recognized that patients value benefits beyond clinical outcomes [20]. Several value elements that their framework mentions as important to individual patients are similar to our findings, such as complexity of the treatment regimen, impact of care options on the ability to return to work, and the negative impact of the condition on family and caregivers. ICER has proposed methodologies for including these in assessments. We hope our set of elements provides a basis for further efforts in developing and standardizing methods such as these.
It is important to consider how the patient-informed value elements fit within the current discussion of patientcentered value assessment across jurisdictions. Cost-effectiveness analyses traditionally reflect population-level costs for the average patient experience rather than considering individualized outcomes, sparking recent criticism over this "one-size-fits-all" approach [21]. Garrison et al. [7] posited that a challenge with assessing value from the individual perspective relates to bounded rationality: the limitation and biases that affect a patient's decision making. Others describe patients' healthcare decisions as guided by value traits and life priorities [22]: Armstrong and Mullins [22] described a taxonomy of patient values that encompassed decisional, situational, and external values. The patientinformed value elements such as tolerability, life impact, and treatment costs are decisional values. The patientinformed value elements within the domains of treatment access and life impact reflect situational values. Finally, the social impact domain, along with the patient-informed value elements within the stigma category, reflect external values. This provides some evidence of the external validity of our patient-informed value elements. The recommendations for tailoring, quantifying, and applying the patientinformed value elements offer a range of opportunities that do not impose a "one-size-fits-all" approach. Rather, the patient-informed value elements are a comprehensive guide that can be leveraged to enhance the patient voice in value assessment and make value assessments more patient centric [5]. The ISPOR Task Force Report, with international stakeholder representation, suggests this is a global concern [11]. Recently, changes to value frameworks, such as in the  [20], illustrate this interest in the USA. This is further evidenced by efforts among jurisdictions in the UK and Europe as well [15,[23][24][25].
Our approach is not without limitations. First, while we engaged patient stakeholders from diverse medical communities, it is difficult to account for all possible patient perspectives. For example, the aspect of altruism in protecting the community from disease spread emerged in a recent study of patients with hepatitis C [26]. It may be that the fear of contagion was not prioritized in our patient-informed elements because of a lack of a patient representative from the infectious disease community. We acknowledge this model is not static and will require ongoing refinement with additional patient groups, and additional recommendations will follow as we gain more experience with their application in value assessment. We continue to build our connections, such as in women's health, to bring this important work to other patient communities not represented in our initial effort. This work is a first step in deriving patient-informed value elements. Although one of the SAC patient representatives was from the Hispanic community, we did not engage Spanish-speaking patients in this phase of the work. It is possible that non-English-speaking communities may have different values in healthcare that stem from different experiences. One immediate next step for our team is to conduct similar patient engagement within the Hispanic community using Spanish language instruments. Our experience with the Hispanic community has revealed a limited number of organized communities that advocate on behalf of patients. This has taught us that we must first understand the needs of the community before we can introduce the concept of value assessment. Our team has invested significant effort into understanding where to start, which is enabling replication of this work in the Hispanic community. A similar investment will be needed for other cultural and racial ethnic groups. The patient voice represented in this work reflected individuals already familiar with value assessment. Nonetheless, this work provides a foundation from which to advance the patient voice in value assessment.
The main goal of this work was to identify patientinformed value elements that could be tested and used in existing value frameworks or value assessments, such as cost-effectiveness analyses. It was not the intent to develop a value framework de novo nor to develop a patient-reported outcome instrument. While the patientinformed value elements are an important advancement, we acknowledge that more work needs to be done. Future applications will be to examine the potential for the candidate, patient-driven value elements described here to be used in economic models for target medical conditions. Phelps et al. [27] described that potential value elements may fall into a category of "conceptually feasible but generally impractical" for use in economic evaluations.
Additional work by this team will determine elements that may be impractical for economic evaluations but could be meaningfully applied in other health technology assessments, such as patient preference evaluations. In addition, we plan to quantify the importance of our 42 proposed patient-informed value elements across the range of medical conditions. This will address issues of comparability in priorities for the value of health interventions across medical conditions. For example, value elements important to patients with cancer diagnoses may not be equally important to those living with chronic conditions, such as diabetes mellitus, or to those living with rare diseases. Careful consideration of patient-driven elements and their generalizability across medical conditions will be important for future patient-centered value assessments. We hope this work will stimulate a dialogue on approaches to quantifying patient-informed value in economic evaluations.

Conclusion
The formative work described herein responds to the call for patient involvement in value assessment. We present a methodologic process used to derive patient-informed value elements with continuous patient engagement throughout the process. The hope is that these elements can be implemented in forthcoming research to advance the field of value assessment. The ultimate goal is to improve health technology assessment, data-generation, and decision-making processes.