{'fraction_non_alphanumeric': 0.03706607741107706, 'fraction_numerical': 0.016929218048328462, 'mean_word_length': 5.095807082337606, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Neuropathy experienced by colorectal cancer patients receiving oxaliplatin: A qualitative study to validate the Functional Assessment of Cancer Therapy/Gynecologic Oncology Group-Neurotoxicity scale

BACKGROUND Although oxaliplatin is widely established as a standard treatment in colorectal cancer (CRC), oxaliplatin-induced neuropathy has emerged as a prominent dose-limiting side effect associated with quality of life decrements. Ongoing monitoring and management of neuropathy is important for CRC patient quality of life and adherence to treatment. Therefore, a validated self-reported measure of neuropathy would aid in the management and assessment of oxaliplatin-induced neuropathy in clinical practice and research. We sought to evaluate the content validity of the 13-item Functional Assessment of Cancer Therapy/Gynecologic Oncology Group- Neurotoxicity subscale (FACT/GOG-Ntx) for CRC patients receiving oxaliplatin. AIM To understand the neuropathy experiences of CRC patients and assess content validity of the FACT/GOG-Ntx. METHODS Semi-structured concept elicitation and cognitive debriefing interviews were conducted with 31 CRC patients experiencing peripheral neuropathy from current or previous oxaliplatin treatment. Interview data were analyzed using a constant comparative approach, and data were mapped to the FACT/GOG-Ntx to assess content validity. RESULTS Mean age of the sample was 54 (range 34-82). The sample was primarily Caucasian (84%) and consisted of nearly equal numbers of men and women. Participants described 28 unique neuropathy symptoms; hand tingling (experienced by 87% of respondents); feet tingling (81%); hand numbness (68%); and feet numbness (84%) were most frequently mentioned. Neuropathy symptoms occurring on the feet were most often identified as most bothersome by participants. Eleven of the 13 FACT/GOG-Ntx items exhibited moderate to strong evidence of content validity. Two items related to trouble hearing and ringing in the ears had weak support; however, these items represent severe neuropathy and could be useful for a patient reported outcome measure. CONCLUSION The FACT/GOG-Ntx represents the key neuropathy experiences of CRC patients treated with oxaliplatin.


INTRODUCTION
Although oxaliplatin is widely established as a standard treatment in colorectal cancer (CRC), oxaliplatin-induced neuropathy has emerged as a prominent dose-limiting side effect associated with quality of life decrements [1,2] . CRC patients report significant impairment in their activities of daily living because of their neuropathy, including difficulty with tasks requiring fine motor skills and dexterity, mobility, and balance [3] . Oxaliplatin induced neuropathy may be acute or chronic [1][2][3][4][5][6][7] . Acute neuropathy typically occurs within 1-2 d of the first oxaliplatin infusion [1] . Acute symptoms tend to resolve spontaneously within one week and typically do not necessitate dose reductions; however, symptoms may return with subsequent administration of oxaliplatin [1,8,9] . Chronic neuropathy develops gradually over time and is related to the cumulative oxaliplatin dose [2,5,8,[10][11][12] . Although symptoms can resolve within six months of treatment [13] , there are several reports of chronic neuropathy lasting two years or more [8,10,14,15] . Prevention and treatment options for oxaliplatin-induced neuropathy are limited; approaches include scheduled drug holidays, magnesium and calcium infusions, and pharmacologic interventions including anti-depressant and anti-epileptic agents [1] .
Ongoing monitoring and management of neuropathy is important for patient quality of life and adherence to treatment. A validated self-reported measure of neuropathy would aid in the management and assessment of oxaliplatin-induced neuropathy in clinical practice and research. The Functional Assessment of Cancer Therapy/Gynecologic Oncology Group-Neurotoxicity subscale (FACT/GOG-Ntx) is a 13-item subscale of the FACT-G that was developed with input from the Eastern Cooperative Oncology Group (ECOG), the Gynecologic Oncology Group (GOG), and the National Surgical Adjuvant Breast and Bowel Project. The FACT/GOG-Ntx includes the previously validated 11-item FACT/GOG neurotoxicity subscale [16][17][18][19][20] , which assesses sensory symptoms (e.g., numbness, tingling, and discomfort in hands and feet), motor symptoms (e.g., trouble walking; buttoning buttons), and ototoxicity (e.g., ringing or buzzing in ears). With the introduction of oxaliplatin, and its unique cold hypersensitivity, two new items were written by investigators from National Surgical Adjuvant Breast and Bowel Project. We sought to examine the content validity of the 13-item FACT/GOG-Ntx for CRC patients receiving oxaliplatin.

MATERIALS AND METHODS
Content validity of the 13-item FACT/GOG-Ntx subscale was assessed via concept elicitation and cognitive debriefing interviews with CRC patients experiencing oxaliplatin-induced neuropathy. Trained interviewers used a semi-structured guide that was informed by literature and guides from prior work to assess content validity of PRO measures [21][22][23][24] . The study protocol was reviewed and approved by the Northwestern University Institutional Review Board; all participants provided informed consent.

Participants
Patients were recruited from the Robert H. Lurie Cancer Center and the CRC Alliance (https://www.ccalliance.org/) in 2017. Eligible patients were age 18 and older, had a diagnosis of CRC (any stage), were receiving or had received oxaliplatin, and were experiencing peripheral neuropathy. Patients with a cognitive impairment or those experiencing neuropathy from other causes were excluded. Participants were interviewed in-person or via phone, and were compensated for their time.

Concept elicitation
After providing basic demographic information, patients were asked to list their neuropathy symptoms. Additional details were gathered on each symptom, such as when the symptom began, frequency of the symptom, if it co-occurred with other neuropathy symptoms, and its impact on functioning. Patients rated each symptom as it pertained to their health-related quality of life on a 0 to 10 scale, where 0 = not at all important and 10 = extremely important. Lastly, patients were asked to share which neuropathy symptom was most bothersome, and why.

Cognitive debriefing
After the concept elicitation interview, patients completed the FACT/GOG-Ntx 13 plus 6 additional items ( Table 1). The FACT/GOG-Ntx measures the severity and impact of symptoms of neurotoxicity, such as numbness, discomfort, or trouble with motor skills, over the past 7 d. Responses are selected from a scale of 0 ("not at all") to 4 ("very much"). The 6 additional items were written to test whether numbness and tingling should be combined in one item, as in the current version of the FACT/GOG-Ntx (items NTX1 and NTX2), or divided into separate items (additional items 1-4, Table 1). Additional items 5-6 tested whether "discomfort" or "pain" best fit the patients' neuropathy experiences. After the patient completed the measure and the 6 additional items, the interviewer conducted a cognitive interview with the patient using a structured interview guide based on the work of Willis [25] to assess patients' understanding of the measure's instructions, items, and response options. Patients were asked to state each item in their own words, describe how they arrived at their response, and indicate each item's relevance to their experience.

Statistical analysis
Interviews were audiotaped, transcribed verbatim, and transcripts were de-identified. A list of neuropathy symptoms that patients reported during concept elicitation was compiled and redundant symptoms were removed; this condensed symptom list formed the basis of a codebook. Two experienced qualitative researchers analyzed the concept elicitation data systematically using a constant comparative approach [26] . The researchers met regularly to review data for each code. Saturation-the point at which no new, relevant data emerges [27,28] -was tracked using a saturation tracking table. Summaries of the data for each code were written, highlighting terminology used by patients and their experiences. These summaries were mapped to the FACT-GOG/Ntx content. The mapping process aimed to highlight (1) dimensions of  neuropathy identified by patients and covered by FACT/GOG-Ntx; (2) FACT-GOG-Ntx content that does not align with CRC patients' experiences with oxaliplatininduced neuropathy; and (3) experiences with oxaliplatin-induced neuropathy not represented by the FACT/GOG-Ntx [29] . Although content validity assessment primarily relied on the concept elicitation data, we also utilized the literature and cognitive interview data.

Sample
Sociodemographic and clinical characteristics of the concept elicitation sample (n = 31) are shown in Table 2. Our sample was primarily Caucasian (n = 26, 84%) and consisted of nearly equal numbers of men and women. Approximately one-third of the sample (n = 11, 35%) were currently receiving treatment.

Concept elicitation results: Patient reported symptoms of neuropathy
In response to the question, "Please describe all of the neuropathy side effects that you are currently experiencing," patients reported over 60 side effects (i.e., symptoms of neuropathy). After redundant categories were removed, 28 unique concerns remained ( Table 3). Saturation of patient-reported neuropathy symptoms occurred at interview number 31. The most frequently mentioned symptoms were associated with the hands and feet: hand tingling (experienced by n = 27, 87% of respondents); feet numbness (n = 26, 84%); feet tingling (n = 25, 81%); and hand numbness (n = 21, 68%). Additionally, 74% (n = 23) of participants reported cold sensitivity in their hands or feet, which they described as feelings of shock, stinging, or pain upon touching cold items. Neuropathy symptoms affecting the feet were most frequently identified as most bothersome ( Table 4). Seven of the 28 (25%) respondents to this question stated that numbness in the feet was most bothersome. For example, patient 014 said, "My feet have been most bothersome, because with the numbness, you know, especially if I'm getting tired, I can't feel my feet and sometimes I just find it difficult to start walking in the morning." Five of 28 (18%) stated that discomfort/pain in the feet was most bothersome.  not all patients completed the entire cognitive interview. Thus, the sample size varies slightly across questions, as noted below. Twenty-eight patients provided feedback on the questionnaire instructions; all 28 (100%) said the instructions were clear. For every item, 100% of respondents (28 of 28) indicated that they were confident or very confident in their response to the item. All items were interpreted in ways that were consistent with the intended meaning (results not shown). Twenty-three of 28 participants (82%) said they thought of a specific time period when answering questions; the most common time period referenced by this group were the past 7 d (12 of 23 participants, 52%) or the current day (7 of 23 participants, 30%). Patients were also asked if their responses would have been different if the instructions said to think of the last 24 hours. The majority of patients (18 of 28, 64%) said no. Almost all participants (27 of 29, 93%) reported that the instrument captured their experiences with treatment-related neuropathy. Cognitive interview results related to the relevance of individual items is provided below and in Table 5, as part of the content validity assessment.

Content validity for FACT-GOG-Ntx items
Items NTX1 -I have numbness or tingling in my hands: Numbness of the hands was commonly described as a lack of feeling or sensation that affected activities of daily living such as buttoning buttons, feeling and holding objects, and writing (Supporting quotations are shown in Table 5). Tingling of the hands was commonly described as a feeling of pins and needles, stabbing, or prickling. Patients reported that the numbness or tingling of the hands was often a constant sensation. During the cognitive interview, 26 of 28 patients (93%) said that item Ntx1 was relevant to their experience.
Item NTX2 -I have numbness or tingling in my feet: Patients described numbness in their feet as a lack of feeling or sensation. Tingling was commonly described as a feeling of electricity, vibration, or being asleep. Patients reported that their numbness or tingling was often a constant sensation that impaired mobility and affected daily  28 patients (7%) thought that the concepts of numbness and tingling should be separated into two questions. For both NTX1 and NTX2, 100% of patients (28 of 28) indicated they were confident or very confident in their ability to provide a response to the items as they were written; none of the patients found the questions to be confusing. When asked which questions were difficult to answer, patient 031 identified the additional questions 3-4, which separated numbness and tingling in the feet, as most difficult to answer: "Some of the neuropathy questions (were difficult) where you had separate sensations but they should go together, like tingling and numbness." Item NTX3 -I feel discomfort in my hands: Over one third of patients (12 of 31, 39%) listed hand discomfort or pain as a symptom of their neuropathy during concept elicitation. Hand discomfort tended to interfere with everyday activities or "basically anything that involved my hands" (patient 033). Twenty-five of 28 (89%) cognitive interview participants said item NTX3 was relevant to their experiences.
Item NTX4 -I feel discomfort in my feet: Eighteen of 31 participants (58%) listed symptoms consistent with discomfort in the feet, including aches, pain, burning sensations, and abnormal sensations. Discomfort in the feet affected activities such as standing and driving. Patients also described feeling as if a small object was in their shoe or under their foot, which affected balance and walking. Twenty-five of 28 (89%) cognitive interview participants said the item was relevant to their experiences.
Discomfort vs pain: Patients overwhelmingly stated that "discomfort" was more consistent with neuropathy experienced in their hands than "pain" (23 of 28 patients, 82%). Likewise, 21 of 28 patients (75%) said that "discomfort" was more consistent with peripheral neuropathy experienced in their feet than "pain". Patients described discomfort as general, more constant, less severe and less likely to interrupt daily activities than pain. Pain was described as throbbing, hurting, more severe, and more likely to stop daily activities. According to patients: "Pain came sometimes but discomfort was always there." (PT 011) "I have more discomfort than pain." (PT 019) "Because my experience is numbness and tingling which is not painful, doesn't hurt, (it is) just annoying." (PT 023) "It is irritating as opposed to hurting." (PT 025). Mean responses on the discomfort and pain items are shown in Figure 1. Thirteen of 28 patients (46%) had identical responses to the items referencing discomfort in the hands and pain in the hands. Of the 15 patients (54% of the sample of 28) whose responses were different, all 15 reported more hand discomfort than hand pain. Moreover, of these 15 patients, 11 (73%) reported "not at all" in response to "I have pain in my hands" while reporting levels of hand discomfort ranging from "a little bit" to "very much". Fewer patients (11 of 28, 39%) had identical scores for foot discomfort and foot pain. Sixteen of the 17 (94%) patients with differing scores reported more foot discomfort than pain. We investigated the significance of differences in mean responses using two-tailed t-tests assuming equal variance. The findings revealed significant group differences for NTX3 and Add-5 (P = 0.007) and NTX4 and Add-6 (P = 0.016).
Item NTX5 -I have joint pain or muscle cramps: Seven of 31 participants (23%) listed symptoms consistent with joint pain or muscle cramps. These symptoms were impactful to patient quality of life. For example, according to patient 023, "It was my Retain the item. Strong support in concept elicitation data, cognitive interview data, and extant literature.

NTX10
I have pain in my hands or feet when I am exposed to cold temperatures Strong "It's just a…ultra-sensitivity to cold, anything, it was, it's like anything colder than my body temperature would either cause like pain or I couldn't hold, like in my hands, I wouldn't be able to hold things...And then for my feet…if I go into cold water or if I'm outside in the cold and my feet seem to get cold faster first before anything else, so...the cold sensitivity would be like touching something extremely hot, like you…your body reacts to pull away, you know. And then for my feet, it's actually painful when they get cold, when they're exposed to cold" (PT 012).
Retain the item. Strong support in concept elicitation data, cognitive interview data, and extant literature.

NTX11
I have difficulty breathing when I am exposed to cold temperatures Moderate "As the temperature of the air started to change, to breathe in was difficult-it became kind of painful even" (PT 006 calves that were cramping and it was, oh, I don't think I'd call it severe, but it was at least moderate to severe cramping. And it was pretty painful." Over half of cognitive interview participants (15 of 28, 54%) reported the item as relevant to their experience.
Item HI12 -I feel weak all over: Five of 31 concept elicitation participants (16.1%) mentioned feeling weak. Feeling weak was used to describe neuropathy symptoms in the feet, hands, legs, and feet. Others noted weakness when describing the impact of neuropathy symptoms, such as difficulty walking. For example, "Sometimes I have weakness in general, like I'll be walking and I feel like I'm going to trip" (patient 010). Twenty-two (78.6%) of cognitive interview participants said the item as relevant to their experience.
Item NTX6 -I have trouble hearing: Trouble hearing was not spontaneously mentioned as a neuropathy symptom in the concept elicitation data. Some support for this item came from the cognitive interviews; three of 28 (11%) participants reported the item "I have trouble hearing" as relevant. Of the 25 who said the item was less relevant, 22 did not have trouble hearing, and three reported having hearing problems prior to beginning treatment.

Item NTX7 -I get a ringing or buzzing in my ears:
Ringing or buzzing in the ears was mentioned spontaneously by one patient in the concept elicitation interview. A quarter of cognitive interview participants (7 of 28, 25%) reported the item as relevant to their experiences. Of the 21 (75%) who said it was not relevant, 20 did not experience ear ringing or buzzing, and one was unsure if his ear ringing was a neuropathy symptom.
Item NTX8 -I have trouble buttoning buttons: Seven of 31 participants (23%) described limited fine motor function during concept elicitation that affected their ability to button buttons. The process of buttoning buttons was described as time consuming and frustrating, causing some to require assistance or avoid wearing clothing with buttons. "It (buttoning buttons) took more time and it was frustrating having fine motor function limited" (patient 020). A majority of cognitive interview participants (25 of 28, 89%) reported the item as relevant to their experience.
Item NTX9 -I have trouble feeling the shape of small objects when they are in my  Patients' comments about trouble walking ranged from fear of falling, practicing caution while walking, and having to limit or avoid activities, such as exercise. Twenty-four of 28 (86%) cognitive interview participants stated that the item was relevant to their experience.
Item NTX10 -I have pain in my hands or feet when I am exposed to cold temperatures: Twenty-three of 31 concept elicitation participants (74%) reported hand or feet pain when exposed to cold temperatures. Some likened the pain to being shocked, "If I touch cold things, I get little zings going through my fingers. It intensifies if it's colder. Like if I pull something out of the freezer, sometimes I have to drop it because it's too cold" (patient 010). Patients reported limiting exposure to cold items and temperatures because of the pain and discomfort. All 28 (100%) cognitive interview participants reported the item as relevant to their experiences.
Item NTX11 -I have difficulty breathing when I am exposed to cold temperatures: Three of 31 concept elicitation participants (10%) listed difficulty breathing when exposed to cold temperatures as a symptom of neuropathy. They described feeling their throat close, throat spasms, or choking when exposed to cold temperatures. They also described feeling like needles were sticking in their throat. While relatively few patients listed difficulty breathing when exposed to cold temperatures in the concept elicitation interview, those who reported the symptom rated its importance to quality of life high (mean score=8). Moreover, 16 of 28 (57%) cognitive interview participants reported the symptom as relevant to their experience.
Coverage of all patient-reported concepts on the instrument: To ensure that the instrument adequately covers symptoms of importance to patients, we considered whether symptoms reported by at least 20% of the sample (Table 3) during concept elicitation were represented in the FACT/GOG-Ntx. Nine symptoms fit this criteria: hand tingling, feet tingling, hand numbness, feet numbness, cold sensitivity in the hands or feet, discomfort in the feet, discomfort in the hands, impaired fine motor skills, and joint pain or muscle cramps. Each of these symptoms is included in the FACT/GOG-Ntx.

DISCUSSION
In this qualitative study with 31 CRC patients experiencing peripheral neuropathy, tingling and/or numbness of the hands and feet were the most commonly experienced peripheral neuropathy symptoms. Additionally, almost 3 out of 4 participants reported cold sensitivity in their hands or feet. Neuropathy symptoms affecting the feet were most bothersome to patients. The qualitative concept elicitation data, in combination with data from cognitive interviews and the literature, provide moderate to strong support for the content validity of 11 the 13 items of the FACT/GOG-Ntx-13. Two items -"I have trouble hearing" (NTX6) and "I get a ringing of buzzing in my ears" (NTX7), had limited support in our data. However, limited support is not surprising given that hearing impairment is a symptom of severe neuropathy, and oxaliplatin therapy may be discontinued or reduced prior to impacting hearing. We recommend retaining these items as indicators of severe neuropathy.
Patients related more with the term "discomfort" than "pain" when reporting neuropathy in their hands and feet. Quantitative responses to items with either "discomfort" or "pain" showed that patients report higher levels of discomfort relative to pain. These results are consistent with the original selection of the term "discomfort" over "pain" owing to the observation that discomfort is reported earlier than pain in the trajectory of emerging neuropathy. These findings also highlight the need for developers of other patient reported outcome measures to consider whether pain or discomfort is the best concept for their particular population or condition. We also considered whether including "numbness or tingling" in a single item was problematic due to the possibility that such items would be "double-barreled." Most patients reported experiencing the two symptoms together. Patient comments suggest that numbness and tingling tend to appear, increase, and decrease in similar ways; future work should confirm this observation. Most patients did not find the items on "numbness or tingling" in hands or feet to be confusing, although they did understand the distinction between the two symptoms. Only 2 patients (7.1%) stated that the concepts should be separated into two questions. Adding questions to the Ntx-13 that separate numbness and tingling is not likely to improve the measure or change responses to any measurable degree. Patients most often considered the last 7 days when responding to the questionnaire. The 7 d recall period is consistent with other PRO measures recommended for use in clinical oncology [30] .
This study has a number of strengths, including a relatively large concept elicitation sample and a cognitive interview protocol that closely examined patient interpretation of key concepts. Our methods are consistent with Food and Drug Administration Guidance and other published guidelines for assessing content validity [29,31] .

Research background
A content valid assessment of neuropathy is needed for clinical research among colorectal cancer (CRC) patients receiving oxaliplatin. The authors assessed the content validity of the Functional Assessment of Cancer Therapy/Gynecologic Oncology Group-Neurotoxicity (FACT/GOG-Ntx) scale among CRC patients who had received oxaliplatin. The measure exhibited good content validity. Moreover, patients reported that foot neuropathy was most bothersome for them. This study is significant because the authors provide evidence that the FACT/GOG-Ntx is suitable for use in clinical trials and other research studies of this population.

Research motivation
This study examines the neuropathy experiences of CRC patients and the appropriateness (i.e., content validity) of a patient reported outcome measure of neuropathy. These topics are important to examine because patient reported outcome measures are needed to assess drug side effects in clinical trial settings.

Research objectives
The main objective was to test the content validity of FACT/GOG-Ntx. This objective was realized; the measure was found to have content validity and can be used in future research and clinical practice.

Research methods
The authors used semi-structured patient interviews to assess the FACT/GOG-Ntx. Semistructured interviews entail using a set list of questions, administered by a trained interviewer. Interviews typical contain a combination of closed-ended and open-ended questions. By using pre-planned and spontaneous probing questions, the interviewer is able to gather a detailed description of the key topics from the perspective on the interviewee.

Research results
The qualitative concept elicitation data, in combination with data from cognitive interviews and the literature, provide moderate to strong support for the content validity of 11 the 13 items of the FACT/GOG-Ntx-13. Two items -"I have trouble hearing" (NTX6) and "I get a ringing of buzzing in my ears" (NTX7), had limited support in our data. However, limited support is not surprising given that hearing impairment is a symptom of severe neuropathy, and oxaliplatin therapy may be discontinued or reduced prior to impacting hearing. The authors recommend retaining these items as indicators of severe neuropathy.

Research conclusions
The FACT/GOG-Ntx has content validity for CRC patients receiving oxaliplatin. Patients related more with the term "discomfort" than "pain" when reporting neuropathy in their hands and feet. The FACT/GOG-Ntx has content validity for CRC patients receiving oxaliplatin. This study builds upon the body of evidence supporting the use of the FACT/GOG-Ntx in future research and clinical practice.

Research perspectives
Existing patient reported outcome measures can be tested for their validity in new, specific populations. The authors anticipate continued advancement in the use of patient reported measures in clinical research and in drug development. Future work on the use of patient reported outcomes measures in clinical practice is best suited for a combination of patientfocused, qualitative research and large, quantitative surveys to assess measurement properties.