{'fraction_non_alphanumeric': 0.029423459244532803, 'fraction_numerical': 0.00805168986083499, 'mean_word_length': 5.361930860033727, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Development of a Patient-Reported Outcome Measure for Non-Alcoholic Steatohepatitis (NASH-CHECK): Results of a Qualitative Study

Background Non-alcoholic steatohepatitis (NASH) is a progressive form of non-alcoholic fatty liver disease characterized by excessive liver fat accumulation, inflammation, cell injury, and fibrosis. It is viewed as largely asymptomatic in its earlier (non-cirrhotic) stages, and information on the patient-perceived impact of NASH is scarce. Objective This study aimed to develop a NASH-specific patient-reported outcome (PRO) measure (NASH-CHECK) for use as a trial endpoint, using methods compliant with regulatory expectations. Methods A NASH conceptual model was developed based on the literature and clinical/patient expert review. The model guided concept elicitation (CE) interviews in patients with non-cirrhotic NASH recruited via a US tertiary care center. NASH-CHECK content was generated via thematic analysis of CE data and review by clinical/patient experts. Cognitive debriefing (CD) interviews with US patients evaluated content validity. Results The literature review confirmed that NASH impacts on functioning and health-related quality of life (HRQoL). Overall, 23 CE and 20 CD interviews were conducted. Key symptoms reported in CE interviews included pain in the upper-right abdomen (n = 14), fatigue (n = 18), poor sleep quality (n = 12), impaired memory (n = 13), and reduced focus (n = 11); key HRQoL impacts included impaired physical functioning, reduced ability to conduct daily living tasks, reduced quality of relationships, low mood, anxiety, and self-consciousness. The 52-item first-draft NASH-CHECK was reduced to 31 items based on patient feedback on item relevance, acceptability, and comprehension. Conclusions The interviews revealed key symptoms and broad HRQoL impacts of NASH. As a disease-specific PRO measure assessing symptoms and HRQoL, the NASH-CHECK is relevant, comprehensive, and acceptable to patients and clinicians.


Introduction
Non-alcoholic fatty liver disease (NAFLD), the most common chronic liver disease, is characterized by a build-up of fat in the liver in the absence of excessive alcohol consumption [1][2][3]. NAFLD occurs in the context of metabolic syndrome, being prevalent among patients with type 2 diabetes mellitus, obesity, and cardiovascular diseases [4,5]. Approximately 25-30% of patients with NAFLD will develop non-alcoholic steatohepatitis (NASH), the progressive and most severe form of NAFLD [6]. Progression to NASH is characterized by inflammation of the liver, with evidence of liver cell injury and different degrees of scarring or fibrosis [7,8]. NASH can lead to progressive liver cirrhosis and eventual liver failure, with associated morbidity and mortality [9,10].
NASH is often considered an asymptomatic, "silent" disease in its early stages; however, symptoms such as chronic fatigue, malaise, apparent hepatomegaly, and upper-right quadrant abdominal fullness and discomfort have been reported [3,11,12]. Because the condition is viewed as largely asymptomatic, published literature on the patientperceived impact of NASH has been scarce, although recent evidence suggests that patients with NASH experience symptom burden and impaired health-related quality of life (HRQoL) [13][14][15][16][17]. Histologic surrogate endpoints (such as reduction of hepatic fibrosis, inflammation, and hepatocellular ballooning) or non-invasive measures (magnetic resonance-measured hepatic fat content and serologic biomarkers for early phase trials) are most often included as

Key Points for Decision Makers
Non-alcoholic steatohepatitis (NASH) is often considered an asymptomatic, "silent" disease in its early stages, but evidence suggests that patients experience symptoms, including fatigue, malaise, and abdominal discomfort, that can impair health-related quality of life (HRQoL).
As reported in qualitative interviews with patients, symptoms of non-cirrhotic NASH include abdominal pain, fatigue, poor sleep, and cognition problems. NASH affects patients' physical, social, and emotional functioning and ability to perform daily activities.
A new patient-reported outcome measure of NASH symptoms and HRQoL, the NASH-CHECK has been developed. The NASH-CHECK is relevant, comprehensive, acceptable to patients, and suitable for use in clinical trials and clinical settings to evaluate patientperceived impact of NASH.
primary endpoints in clinical trials in NASH [18][19][20]. While these endpoints present crucial information on the histological manifestations of the disease to clinicians, they do not reflect the impact of the disease and the treatment outcomes that matter most to patients.
Patient-reported outcome (PRO) measures (PROMs) are standardized means of evaluating symptoms, functioning, HRQoL, and the wider impact of a disease from the patient perspective. PROs add value by complementing key clinical endpoints in clinical trials [21]. Such information is vital in characterizing patients' views on the effectiveness of an intervention. In addition, PROMs are useful for monitoring patients in clinical practice, allowing physicians to evaluate patients' symptom experiences and changes with treatment alongside physician-rated assessments [21]. PRO data are viewed as important by major regulatory authorities to capture the patient perspective in the drug development process [22]. As such, the US FDA has provided guidance for evaluation of PROMs intended for label claims, and the European Medicines Agency (EMA) has issued a reflection paper on the use of HRQoL measures in the drug evaluation process [23,24]. The quality standards outlined therein emphasize the need for input from the specific target population during development of a PROM and evidence of content validity based on cognitive debriefing (CD) interviews [23,24]. However, few of the currently available PROMs for NASH meet these criteria [25].
The objective of this study was twofold: (1) to understand the humanistic burden and day-to-day impact of living with non-cirrhotic NASH and (2) to use this information to develop a novel PROM to assess symptoms and HRQoLrelated impacts of NASH that aligns with regulatory guidance. The measure will facilitate understanding of the symptoms and wider impact of NASH from the patient perspective and will provide a valuable complement to support key clinical endpoints in NASH.

Materials and Methods
An international NASH PRO Task Force comprising PRO researchers, patient advocacy representatives, clinical experts, and representatives of the pharmaceutical industry was established to explore the humanistic burden of NASH and to oversee the development of the NASH-CHECK PROM. Institutional review board approval was granted for the study by the Western Institutional Review Board (Puyallup, Washington).
Development of the NASH-CHECK PROM followed FDA guidance for PROMs intended for use in regulatory label claims and also was consistent with the EMA's reflection paper on HRQoL measures [22][23][24]. The FDA guidance recommends that the development of a PROM should be based on qualitative research conducted with the target population to establish the content validity of the measure. The measure should then undergo a quantitative (psychometric) evaluation to explore its measurement properties [23]. Qualitative development of the NASH-CHECK, consistent with FDA guidance, included three phases to establish the content validity of the measure: (1) development of a draft conceptual model based on literature search and discussions with medical experts; (2) content generation through concept elicitation (CE) interviews with patients diagnosed with NASH and fibrosis levels F1-F3 based on the draft conceptual model; and (3) evaluation of content validity through CD interviews with the same target population [23]. Specifically, the CE interviews explored the patient-perceived impact of living with NASH and informed item generation. The CD interviews evaluated the appropriateness and comprehensibility of the measure. Future planned analyses will explore the psychometric properties of the measure.

Literature Review
A targeted literature review was conducted using MEDLINE and Embase to identify articles published between 1996 and 2016 relating to the symptomatic and humanistic burden of non-cirrhotic NASH. Additionally, the review sought to identify validated PROMs developed specifically for this patient population. The targeted review was supplemented by a manual search of gray literature (general internet searches and congress websites), patient blogs, and patient group websites, including the British Liver Trust and the Global Liver Institute, to search for qualitative information on patients' experiences of living with non-cirrhotic NASH [26,27]. Areas of patient-reported symptoms and impact on daily life and HRQoL associated with NASH from the published and gray literature searches and social media sites were summarized and reviewed with clinical experts and patient advocacy representatives. These results were used to develop an initial conceptual model for NASH

Study Participants
Patients were recruited in the USA from a tertiary care center in Richmond, Virginia, and the Texas Liver Institute, San Antonio, Texas. A purposive sampling approach was used to achieve a sample size sufficient to support qualitative analyses and concept saturation [28]. For this study, non-cirrhotic patients were targeted, since the common and often severe impairments associated with cirrhosis of any cause have the potential to overwhelm any HRQoL findings that may be specific to NASH.
The inclusion criteria were adults aged 18-75 years with liver biopsy-confirmed NASH with fibrosis levels F1, F2, or F3 based on the NASH Clinical Research Network histological scoring system [29] or phenotypic diagnosis (i.e., alanine transaminase levels [≥ 60 IU/L for males or ≥ 40 IU/L for females] and body mass index [kg/m 2 ] [≥ 23 in Asian individuals or ≥ 27 in non-Asian individuals]) and type 2 diabetes mellitus. Patient recruitment from the Texas Liver Institute had the additional inclusion criterion of Hispanic ethnicity.
The exclusion criteria were patients with other liver disease, other serious comorbid conditions, clinical evidence of cirrhosis, excessive alcohol consumption (defined as more than 20 g per day in females and more than 30 g per day in males, on average, for a period of more than 3 consecutive months within 1 year prior to the study), active substance use, severe mental illness, extreme obesity, or poorly controlled diabetes (glycated hemoglobin [HbA1c] levels ≥ 9.5% [80 mmol/mol]).

Concept Elicitation Interviews
Each qualitative CE interview with the eligible NASH patients was led by two experienced qualitative researchers and followed a semi-structured interview guide based on the draft conceptual model. The interviews explored patient experiences of NASH symptoms and the impact of NASH on the way patients feel and function. All interviews were conducted in person at a facility close to the recruiting center and were audio recorded and transcribed. Each CE interview lasted approximately 1 h. Patient sociodemographic data were obtained from a patient-completed background questionnaire and medical data from patient records. Electronic Supplementary Material (ESM) 1 presents additional details about the CE interview methodology.

Concept Elicitation Analyses
Data analysis of the anonymized CE interview transcripts and the interviewers' field notes were performed using ATLAS.ti qualitative analysis software (ATLAS.ti Scientific Software Development GmbH; Berlin, Germany). Thematic analysis was conducted by two independent researchers to evaluate key areas of symptomatic and HRQoL impact from the patient perspective [30]. An initial coding frame-comprising topic areas, concepts, and specific symptoms and impacts-was developed based on the interview guide. This coding frame was applied to the data by assigning codes to segments of text in the interview transcripts. Interview transcripts were analyzed in sets of five. The coding frame was modified progressively to accommodate information arising from the analysis. Thematic saturation was reached when no new symptoms or impacts were identified during successive transcript sets. The size of the sample was driven by thematic saturation [28]. Thematic analysis of the data from the CE interviews was used to derive the final conceptual model.

NASH-CHECK Development
The NASH-CHECK PROM was developed based on the conceptual model for NASH. Item generation was guided by the conceptual model and the qualitative data from the CE interviews. An initial item pool was presented to the NASH PRO Task Force, during which the symptoms and HRQoL impacts identified were reviewed and the draft item set was selected. The Task Force also agreed on the instructions, question format, response options, and recall period for the draft measure.

Cognitive Debriefing Interviews
Content validation was evaluated through three rounds of CD interviews with NASH-diagnosed patients according to the inclusion and exclusion criteria. The first round of interviews was designed to identify any initial problems with wording of instructions, items, and response options and to select between alternative formulations of individual items. The second and third rounds were intended to assess the adequacy of changes made to the measure. The purpose of all interviews was also to assess the comprehensibility, understandability, and relevance of the instructions, items, and response options for the NASH-CHECK. Each interview was conducted by two PRO researchers with expertise in qualitative research using a semi-structured interview guide. Rounds 1 and 2 of the interviews were conducted with patients recruited from the center in Richmond, Virginia, and round 3 was conducted with US-Hispanic patients recruited from the center in Texas. All interviews were conducted in person, were audio recorded and transcribed, and lasted approximately 45 minutes. Patient sociodemographic data were obtained from a patient-completed background questionnaire and relevant medical data from patient records. ESM 1 presents additional details about the CD interview methodology.

Cognitive Debriefing Analyses
Analysis of the CD interview data was based on a combination of the interview field notes and transcripts. Participants' feedback on the suitability and relevance of the draft NASH-CHECK PROM, including issues identified with specific elements (instructions, recall period, response options, and individual items), was collated and summarized to inform the measure's revision.

Literature Review
The literature review explored the availability of PROMs targeted toward the NASH population. PROMs that have been used in NASH studies include the 36-item Short-Form Health Survey (SF-36) [31], the Chronic Liver Disease Questionnaire (CLDQ) [32], and an NAFLD-specific version of the CLDQ (CLDQ-NAFLD) [33]. The content of the SF-36 and CLDQ are not specific to NASH and may miss issues of importance to NASH. Moreover, while the CLDQ-NAFLD includes items developed specifically to capture the impacts of NAFLD, neither it nor any other measure to date has followed guidance for the development and validation of a PROM suitable for consideration for regulatory label claims [23,24]. The review of the published literature identified few published studies (n = 6) assessing HRQoL of a biopsyconfirmed NASH population only. Studies presented burden of NAFLD across the spectrum of severity, with NASH sometimes reported as a subgroup; moreover, the terms NASH and NAFLD were not always clearly differentiated. Nonetheless, published literature on the humanistic burden of NASH, although limited, highlights the negative impact that NASH may have on many aspects of patients' lives [34,35]. Patients with NASH report significantly increased lifetime rates of depression and anxiety [36], and psychological factors have been found to be linked to weight outcomes for patients with NASH [37][38][39]. Adults with NASH have lower HRQoL based on the SF-36 than the general population [38]. Further, patients with NASH, relative to those with NAFLD without NASH, report significantly poorer SF-36 physical component scores and significantly poorer scores on the vitality, general health, bodily pain, and role limitations subscales [38].
The narratives from searches of social media reinforce the observation that living with NASH can have a profound impact on many aspects of patients' lives, including their ability to conduct day-to-day activities, their personal and social relationships, and, ultimately, their HRQoL [40,41]. Clinical experts and patient representatives reviewed the findings from the searches of the literature and social media and confirmed that the findings largely matched their views of the patient experience. Additional areas of impact raised by patient representatives were economic impact (cost of medication, lifestyle management) and patient behavior (motivation and eating habits). Taken together, the results of the reviews and expert input were used to develop an initial conceptual model for NASH.

Participant Demographics and Clinical Characteristics
CE interviews were conducted with 23 patients; 16 had a NASH diagnosis based on biopsy, and seven had a phenotypic diagnosis of NASH. The CD sample included 20 patients overall; 16 had a NASH diagnosis based on biopsy, and four had a phenotypic diagnosis. Characteristics of both CE and CD participants included in the analysis are provided in Table 1. Eight patients who participated in the CE interviews also participated in the CD interviews. The inclusion of some patients in both stages provides further confirmation on the interpretation of concepts emerging from the CE interview stage and that they were reflected appropriately in the draft NASH-CHECK.
The CE and CD samples were broadly similar in terms of their demographic and clinical characteristics. The most frequently reported comorbidities among all study participants were type 2 diabetes mellitus, hypertension, high cholesterol, and depression.

Patient-Reported Impact of NASH
Analysis of the CE interview data was completed using five sets of transcripts to assess thematic content saturation. Sets 1 through 4 comprised five transcripts each, and the fifth set contained three transcripts. As expected, most themes were identified in the first set of five transcripts analyzed.
No new symptoms or impacts emerged after the fourth set of transcripts, confirming that concept saturation was reached and supporting that additional interviews would have yielded little or no new concepts of relevance for the construction of a new PRO instrument for NASH.
The symptoms of NASH that emerged from the CE interviews included pain in the upper-right abdomen, fatigue or tiredness, poor sleep quality, and cognitive issues such as impaired memory and reduced focus (Fig. 1). Specifically, in detailing the pain associated with NASH, participants mentioned feelings of discomfort and/or ache, sometimes described as "constant" and "chronic". One participant noted: "Constant dull ache in my right upper quadrant that radiates to my back." Participants' description of fatigue included tiredness, exhaustion, and low or no energy levels, even after the completion of normal activities, after a non-strenuous day at work, or after a full night's sleep, noting: "I get very tired … normal activities fatigue me." Participants also described memory issues relating to difficulty retaining and retrieving information, as well as forgetfulness. Poor sleep quality was reported for a range of issues, including frequent awakenings because of the occurrence of pain, difficulty falling asleep, sleeping for short periods of Almost half of participants also reported the occurrence of itching skin, including the experience of itch on their fingers, hands, elbows, feet, and scalp, noting, "I do have itching, often." The broader HRQoL impact of NASH included impaired physical functioning, ability to conduct daily living tasks, reduced quality of relationships, low mood, anxiety, and self-consciousness (Fig. 2). In particular, participants reported difficulty with physical activities, walking, bending, and reaching. According to one participant: "I used to walk 5 miles a day, and I was riding a bicycle during the summer, too. I can't do any of it now. You know, I just feel like … you feel like everything's been deprived from you." Participants also reported difficulty with chores such as sweeping the floor and gardening, with some noting that their ability to complete such tasks had become more difficult. Nearly one-third of the sample reported a negative impact of NASH on relationships, including interactions with family members, particularly children and grandchildren; one participant pointed out, "You're exhausted; it's hard to be sweet granddaddy when you're worn out and so that limits me from the time." Participants also reported the impact of NASH on their social activities, restrictions in their ability to dine out or with friends, or feelings of tiredness. The psychological impact of NASH related to emotion and mood, self-confidence, and self-esteem included participants' feelings of stigma related to having "fatty liver disease", the impact on their appearance, and feelings of low mood. Participants noted how they felt judged by others, particularly by those who had no understanding of the condition. More than a quarter of the sample reported the impact of NASH on their mental health, mentioning feeling anxious and depressed, "It has affected me mentally and physically … I would … shed tears and cry to myself." Finally, participants reported the impact of NASH on their diet, with more than half of the sample making healthy food choices because of NASH and others specifying that their diet had become bland and restrictive. Figure 3 presents the final conceptual model, which was refined based on the CE interview results and in collaboration with clinical and patient advocacy experts. The CE interviews confirmed that patients with NASH experience significant impairment in HRQoL and that patients report aspects of the disease not fully covered by existing PROMs. Refinement of the model informed logical conceptual groupings, including NASH symptoms, activity limitations, and social, psychological, and economic impacts of NASH to be evaluated in planned psychometric analyses.

NASH-CHECK Development
The NASH PRO Task Force selected items for the first draft of the US-English NASH-CHECK measure. The first draft of the NASH-CHECK measure comprised 52 items that were representative of the symptoms and HRQoL impacts reported in the qualitative interviews. Overall, 16 symptom items assessing elements of pain, fatigue, itch, sleep disturbance, and cognition and 36 HRQoL items assessing activity limitations, social impact (social activities and relationships), work impact, psychological impact (emotional and self-esteem), and dietary impact were included. A degree of concept duplication was allowed for some of the items in the first draft of the NASH-CHECK so that patient preferences for item phrasing for these items could be evaluated during the first round of CD interviews. The recall period for the initial item set was 7 days, based on input from clinical experts and patient advocates and the recall periods for PRO measures specific to liver disease, which typically range from 1 to 4 weeks [25]. A 1-week recall was considered appropriate to comprehensively evaluate the symptoms and impacts of NASH while minimizing the potential for recall bias.
Following the first round of CD interviews, the items considered duplicative or irrelevant by patients were removed, and the total number of items was reduced to 34 (ten symptom, eight activity limitations, and 16 emotion and lifestyle items). The PROM was revised further after the completion of the second round of CD interviews. The number of items was reduced to 31 after three emotion and lifestyle items were removed. In addition, minimal changes were made to instructions and response options. For example, "at its worst" was moved from the instructions to the opening of each symptom item. Patients considered the 1-week recall period to be appropriate, and no changes were made to it. Finally, the last round of CD interview supported that the NASH-CHECK items were clear, relevant, comprehensive, and easy to complete. No problems were identified in relation to the instructions, recall period, or response format.
The 31-item pilot version of the NASH-CHECK was included in an interventional phase II study and will undergo psychometric analysis to finalize the structure, explore its measurement properties and responsiveness, and inform the interpretation of scores.

Discussion
This study aimed to explore the patient-perceived impact of NASH and to develop a PROM to evaluate the symptom and HRQoL burden among a real-world sample of patients recruited from clinical practice. A review of the limited published literature on the impact of NASH, consultation with medical and patient advocacy experts, and qualitative CE interviews revealed numerous symptoms and impacts associated with NASH that have a wide impact on patients' dayto-day lives, challenging the notion that NASH is a "silent" disease. Key symptoms reported included pain, particularly in the upper-right quadrant, fatigue, itch, sleep disturbance, and cognitive issues. Key HRQoL impacts included activity limitations, social functioning, psychological issues, work impact, and dietary restrictions. Although patients varied in the number of symptoms and impacts reported, a majority experienced multiple symptoms and HRQoL impacts. Only one patient who had been diagnosed with NASH as part of routine medical screening reported that their NASH was completely asymptomatic with no broader HRQoL impacts experienced.
Development of the NASH-CHECK aligned with regulatory expectations [22][23][24] and good research practices [40]. The FDA guidance makes a number of recommendations for PROs that are to be used for label claims. These include the development of a conceptual model to guide development activities, clear documentation of the process of item development and refinement, and the evaluation of content validity via qualitative studies to confirm that the items and domains covered by a PRO are appropriate and comprehensive. The development activities of the NASH-CHECK were guided by a clear conceptual model that was refined during the development process. Input from patients with NASH was central to development of the NASH-CHECK content. Items for the measure were derived from qualitative interviews, during which content saturation was achieved, and CD interviews were used to ensure the suitability of the measure for patients with NASH. Specifically, in the CD interviews, participants reported the NASH-CHECK measure to be relevant, acceptable, understandable, and clear. The instructions, response options, and recall period were considered appropriate, and the recall period is consistent with recommendations from the FDA guidance against lengthy recall periods [23]. Minor changes were made to some instructions and item wording to improve clarity. The structure and concepts measured by the NASH-CHECK measure were maintained during refinement of the measure. The final NASH-CHECK measure was reduced to 31 items (ten symptom items and 21 HRQoL items) based on patient preferences for item relevance, acceptability, and comprehensibility.
Findings from this study also will support future patient-centered research initiatives in NASH. The use of NASH-CHECK with patients will improve understanding of the humanistic burden associated with NASH. Additional analyses are planned to finalize the dimensional structure and explore the psychometric properties of the NASH-CHECK.
Some limitations of this study must be considered. The participants in the CE and CD interviews were sampled from two clinical sites in the USA. A non-probability sampling approach was used, which has the potential to introduce bias because the sample may not be reflective of the wider population. Participation in the interviews was entirely voluntary, and it is possible that such self-selection could also lead to selection bias. Nevertheless, results from preliminary qualitative research conducted with a geographically diverse sample of 35 patients with NASH in North America, South America, Europe, Asia, and Australia are aligned with our findings, in that patients reported similar impacts of NASH [14,42]. In addition, ten CE interview participants and six CD interview participants had grade F3 fibrosis, at the upper end of the severity range of non-cirrhotic NASH. We considered that patients with advanced fibrosis were more likely to have symptoms, and it was important that the sample included a sufficient proportion of these patients to adequately capture their experiences with NASH. Finally, while most participants had biopsy-confirmed NASH, some participants had phenotypic diagnosis (seven CE interview participants and four CD interview participants), which, in the absence of a clear gold standard for clinical biomarkers for NASH, is generally accepted by the clinical community. These patients were recruited from a specialty center, conferring reasonable confidence in their NASH diagnosis.

Conclusion
This study revealed key areas of the symptomatic and broader HRQoL impacts of NASH, informing the development and establishing the content validity of the NASH-CHECK. The NASH-CHECK is a disease-specific PROM assessing symptoms and HRQoL that have been found to be relevant, comprehensive, and acceptable to patients and is suitable for use in both clinical trials and routine practice. Future analyses are planned to explore its psychometric properties and validate the measure for additional populations, including individuals with cirrhotic NASH.