{'fraction_non_alphanumeric': 0.02760023242300988, 'fraction_numerical': 0.008497966298663567, 'mean_word_length': 5.372830363341819, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
From Qualitative Work to Intervention Development in Pediatric Oncology Palliative Care Research

Qualitative methods can be particularly useful approaches to use with individuals who are experiencing a rare disease and thus who comprise a small sample (such as children with cancer) and are at points in care that few experience (such as end of life). This data-based methods article describes how findings from a qualitative study were used to guide and shape a pediatric oncology palliative care intervention. Qualitative data can lay a strong foundation for subsequent pilot intervention work by facilitating the development of an underlying study conceptualization, providing recruitment feasibility estimates, helping establish clinically meaningful inclusion criteria, establishing staff acceptability of a research intervention, and providing support for face validity of newly developed interventions. These benefits of preliminary qualitative research are described in the context of this study on legacy-making, which involves reports of children (7-12 years of age) living with advanced cancer and of their parent caregivers.

In the United States, about 11 630 children under 15 years of age are expected to be diagnosed with cancer in 2013 (American Cancer Society, 2013). Fortunately, childhood cancer accounts for less than 1% of all cancer diagnoses, and about 80% of children with cancer survive 5 years or beyond. However, 1310 children are expected to die from cancer in 2013, representing a relatively rare yet significant and compelling population at potentially high risk for substantial physical, psychological, social, and spiritual suffering. The significance is further magnified by the possible long-term effects that a child's cancer illness and death can have on parents, siblings, extended family members, peers, and health care providers.
Qualitative methods can be particularly useful when data or information is limited and a greater understanding is desired, such as learning from individuals who are experiencing a rare disease (eg, children with cancer) and who are at points in care that few experience (eg, end of life). Qualitative research has traditionally has been described as methods "providing rich data that may be used to generate theory and patient-centered outcome measures, to influence current standards of care, and to ensure the acceptability of behavioral interventions" (Meissner, 2011, p. S83). The use of qualitative methods is also applicable for the evaluation and understanding of new phenomena (Polit & Beck, 2012), development and evaluation of behavioral interventions (Lewin, Glenton, & Oxman, 2009), evaluation of behavioral interventions with understudied patient populations (National Institutes of Health, 2006), and evaluation for whom the efficacy of an intervention is most beneficial (Sandelowski, 1996). Oftentimes, findings from qualitative studies provide preliminary evidence to support randomized clinical trials using complex interventions, especially when the intervention involves active participation by the target patient population (Murtagh et al., 2007). Comprehensive planning and efficacious testing of behavioral interventions is critical to the acceptability of an intervention by the target patient population (Ayala & Elder, 2011;Meissner, 2011). Before introducing the intervention, it is best to have input from those who will be receiving or using the intervention. Their input frequently is gained from a participatory study using a qualitative approach.
Over the past decade, the number of descriptive, exploratory, and retrospective studies on palliative and end-of-life care for children with cancer and other lifethreatening conditions has increased (eg, Foster et al., 2009;Pritchard et al., 2008). However, most studies have focused on understanding and improving the training of clinicians or nurses (eg, Baughcum, Gerhardt, Young-Saleme, Stefanik, & Klopfenstein, 2007;Hale, Long, Sanderson, Carr, & Tomlinson, 2006;Long, Hale, Sanderson, Tomlinson, & Carr, 2008). Only a limited number of such studies have included reports directly from the ill child and/or the child's parents (Hinds et al., 2007). Although the National Institute of Nursing Research (2011) has highlighted the need to move the field of palliative care toward intervention research, gaps still exist for evidence-based interventions to decrease stress on parents and suffering of children with lifethreatening conditions such as cancer.
Although qualitative methodologies can be useful in many different ways to advance the science of pediatric palliative care, this article shows how a qualitative approach can be used in a focused and very specific manner to help develop and refine evidence-based pediatric palliative care interventions. This is an important methodological priority given the preponderance of qualitative methods used in pediatric palliative care and the compelling need to develop and test interventions to address the care needs of these ill children and their families. Thus, our purpose here is to describe how our research team used a qualitative approach in a pediatric oncology palliative care setting to gather ideas and evaluate the acceptability of a new and innovative legacy-making intervention for parents and their children with cancer. We also describe our experience with enrollment of eligible participants and the decisions made by our research team to modify both our data collection time points and our intervention elements.

Qualitative Methods and Intervention Development
Qualitative data collection involves methods such as observation, interviews (one-on-one or focus groups), documents, and audiovisual materials (Creswell & Plano Clark, 2011). Historically, the primary purpose of using such methods has been to explore what is salient about a specific phenomenon of interest (Polit & Beck, 2012) or to discover the phenomena that influence or are relevant to a social or clinical situation. Researchers may select qualitative methods to understand how individuals or groups may experience an influencing phenomenon to which they interpret and ascribe meaning (Pope, Van Royen, & Baker, 2002) or to gain an understanding of participants' perspectives about the benefits and limitations of receiving an intervention (Monaghan, Sanders, Kelly, Cogen, & Streisand, 2011). Based on the analysis of qualitative data, researchers can be better informed to make ongoing decisions about appropriate study design features that will lay the foundation to guide the planning of the next study, particularly for intervention research development (Monaghan et al., 2011;Polit & Beck, 2012).
Data collected through qualitative methods have become an increasingly valuable resource for researchers when developing behavioral interventions for future clinical trial evaluation (Murtagh et al., 2007). Qualitative research findings can provide preliminary data in support of the efficacy of a new intervention, contribute to further refinement of a new or existing intervention, and suggest potential outcomes that result from delivering a new intervention. Additionally, a critical measure of behavioral intervention research supported by extramural funding agencies (eg, National Institutes of Health) is whether the target population viewed the intervention as relevant, acceptable, and beneficial (Meissner, 2011).
Researchers have demonstrated the ability of ill children to participate in studies that employ qualitative methods. For example, children affected by life-threatening illnesses, phenylketonuria, sickle cell disease, and diabetes have answered open-ended interview questions (Cotton, Grossoehme, & McGrady, 2012;Di Ciommo, Forcella, & Cotugno, 2012;Hawthorne et al., 2011;Kiernan, Guerin, & Maclachlan, 2005), resulting in theory, instrument, and intervention development and confirmation. These examples provide strong support for using qualitative methods to understand perceptions from ill child participants for the purpose of intervention development. Thus, our research team chose to use qualitative data collection methods to guide and shape development of a pediatric oncology palliative care intervention.

Qualitative Approach to Gather Ideas on Legacy-Making
The purpose of our qualitative study was to use child and parent self-reports to gather ideas for developing a legacy-making intervention for children with cancer. Legacy-making, or memory making, can be defined as doing or saying something to be remembered (Foster et al., 2009). Although children's hospitals often offer legacymaking activities to pediatric oncology patients or their family members (Foster, Dietrich, Friedman, Gordon, & Gilmer, 2012), such activities are rarely empirically based. Legacy-making interventions have been tested in adult populations (Chochinov et al., 2005), but gaps remain in testing legacy-making activities in children. Legacy-making has been explored from perceptions of staff and bereaved family members (Foster et al., 2009;Foster et al., 2012), but few studies have explored legacymaking from perceptions of children with life-threatening illnesses such as cancer. We were unable to locate published pediatric legacy-making activities or age-appropriate interventions developed for children with cancer based on child self-reports. Therefore, we obtained institutional review board approval to gather child and parent self-reports that would inform development of a legacymaking intervention.
Previous knowledge on legacy-making from bereaved parent and sibling reports (Foster et al., 2009), staff reports (Foster et al., 2012), and adult legacy-making interventions (Chochinov et al., 2005) suggested that legacy-making was potentially significant and helpful for dying children. However, prospective self-reports from parents and children with cancer were needed to determine (a) if children with cancer seemed interested in legacy-making activities, and (b) if so, what types of activities might they be interested in. We chose a qualitative method in which children (ages 7-12 years) responded to 2 open-ended interview questions after parent consent and child assent. Research assistants gathered the selfreports via individual audio-recorded child interviews in a private room near the outpatient clinic. Interviews were transcribed verbatim. Two researchers independently analyzed data from the open-ended questions through qualitative content analysis (Hickey & Kipping, 1996;LoBiondo-Wood & Haber, 2006). The researchers repeatedly read the transcripts to gain a holistic sense of the data. Similar ideas were clustered, and preliminary categories emerged. The researchers reviewed the initial coding scheme, extracted quotes, and regularly discussed the rationale for emerging categories. Data were reexamined by recoding original transcripts and making mutually agreed upon category changes. We used the findings to develop a pediatric legacy-making intervention, which was further shaped after feedback from one of the parentchild dyads. Results are used here to discuss the benefits of preliminary qualitative research.

Benefits of Preliminary Qualitative Studies
By using the same inclusion criteria planned for a future study to pilot the newly developed intervention, the researcher is in a position to examine feasibility of the proposed recruitment methods and inclusion criteria. The results of our qualitative study showed a potential recruitment challenge regarding identification of eligible children at our institution. Eligible children (a) were between 7 and 12 years of age, (b) had any cancer with a poor prognosis (defined as less than a 60% chance of disease-free survival at 5 years postdiagnosis; this determination was made by attending physicians at the time of diagnosis or at any point during treatment when clinical changes have been noted), (c) were able to speak and understand English and had a parent caregiver able to speak and understand English, and (d) had an absence of cognitive impairment and a parent caregiver with an absence of cognitive impairment. Physicians and nurse practitioners notified the principal investigator of patients who met inclusion criteria. Although the participation rate was high, we experienced difficulty in identifying participants with a poor prognosis because the protocol relied on provider referral for patient names. It took 12 months to enroll 8 eligible participants when we had expected to enroll 2 per month. As a result of this experience, we revised our inclusion criteria for the future pilot study to include children with any prognosis. Expanding our inclusion criteria would allow us to proactively review lists created by the cancer center registry of new diagnoses and relapses for a more feasible identification of potential participants. More important, our expected sample for the future pilot study will include participants with varied prognoses and allow us to prospectively explore if prognosis mediates intervention efficacy. Expanding our inclusion criteria will well-position us to formulate data-based inclusion criteria, especially regarding prognosis and point of illness trajectory, best suited for intervention testing in a future multisite trial.
Qualitative work also allows researchers to assess staff acceptability of studies and willingness to support patient participation. Physicians, nurses, and nurse practitioners are key gatekeepers for investigators approaching potential study participants in coordination with patient clinic visits. In our study, the principal investigator (TFA) presented the project to all providers within the pediatric cancer division before beginning recruitment. Some providers expressed concerns regarding use of sensitive terms (eg, legacy, remember) in the study materials that could imply death was imminent. In response, we added to our protocol the opportunity for parents to see the open-ended child interview questions before using them with their children. The questions were as follows: 1. Some children tell me that they would like to write a few words or make something special to give to someone they care about. (1a) Would you like to write words or make something to give to someone? (1b) What would you like to write or make? (1c) What kind of help would you like? 2. Some children who are ill have told me that they hope their families and friends remember certain things about them. What would you like your family or friends to remember most about you?
A parent in the first family approached questioned the word "remember" in Question 2. She felt this question did not apply to her child because they "still had hope." This experience confirmed the possibility of parents perceiving that certain terms used in study scripts, consent forms, or instruments could imply a death-related study purpose, which was not the intent of our study. Therefore, we revised Question 2 to read ". . . remember (or know) . . ." This allowed for alterative wording in case any other parent caregivers were concerned with us using the term "remember" with their child. We also planned to remove the word "legacy" from study scripts and consent forms for future studies to ensure that imminent death would not be implied. Our qualitative data provided the foundation for an empirically based framework for a pediatric legacy-making intervention, informing intervention content and format (Table 1). For example, we learned that children living with advanced cancer with a poor prognosis did not explicitly express end-of-life legacy concerns as we had anticipated. Rather, children expressed that they wanted others to know or remember (a) their personal characteristics, (b) things they like to do, and (c) their connectedness with others. These categories that emerged from child self-reports were used to create content for the planned intervention that would guide children to develop or further build their desired legacies. Guiding questions were developed (by 2 pediatric palliative care and oncology experts and 1 child development expert) for each of these categories to help children speak about legacyrelated topics. First, the participating children reported that they wanted others to know or remember their personal characteristics, including their name, gender, appearance, and personal traits. One child said, "[I want others to know] If I'm a girl or boy and what my name is . . . what I look like." We used these data to develop ageappropriate guiding questions to invite children to express their unique traits and characteristics (eg, name, age, school, favorite color, food, music). Our qualitative data further revealed that child participants wanted others to know or remember the things that they like to do, including their hobbies and interests. For example, one child shared, "I like to sleep. I like sports. Football." This informed the development of guiding intervention questions related to children's favorite activities or things that make children most happy. Children also reflected on their personal desires to express their connectedness with others, such as telling family members how much they are loved. In turn, we created guiding questions for the intervention to elicit children's expressions about their connections with others (eg, family, friends, teacher, pets) and provide children the opportunity to give messages to special people.
We also used the children's self-reports regarding individualization to develop the format and structure of our intervention (Table 1). All the child participants (N = 8; 100%) supported the idea of writing some words or making something to give to somebody. The children suggested specific activities involving crafts, computers, music, and games and suggested that they would prefer to have the choice of working alone or seeking help of others. Based on participants' suggestions, we selected a digital storytelling format in which the legacy-making intervention would be embedded via scheduled videorecorded child interviews. Video recordings could incorporate the child's favorite activities, crafts, or locations A legacy-making intervention format via digital storytelling was chosen that could incorporate audio, video, music, and photos. This allowed the intervention to be individualized to each child's interests. Children could complete the intervention alone or seek the help of others (eg, their room); family members; or pets. A videographer would edit the video recordings and incorporate photographs and music selected by the child to create a digital story for the child and his or her family. This format in some way incorporated each of the children's recommendations. Digital storytelling would produce a final product that could be viewed and distributed via the computer and could include music and photographs, video, and audio of children's preferred crafts or games. The digital storytelling format would allow participation to be tailored to the unique aspects of each child and family so that children could choose to work alone or request assistance from others and still maintain consistent intervention fidelity. Qualitative approaches can also inform feasibility of participant retention. After developing the legacy-making intervention, the principal investigator contacted all parent caregivers of children (N = 8) who previously participated in the child self-report interviews to invite the children and their primary parent caregivers to participate in a follow-up focus group. The focus group was rescheduled once in an effort to increase participation, but only 1 child and 1 parent participated in the follow-up. Attrition (88%) was due to 2 participant deaths, 1 no-show after verbal agreement from parent via phone, 2 passive refusals, 1 active refusal because the child was not feeling well, and 1 active refusal because of a schedule conflict. This information informed us that group follow-up for this population presented challenges and was not feasible. In response, we designed the subsequent pilot study without group data collection methods but rather individual appointments that could be flexible and scheduled according to the unique needs of each child and family.
One of the greater contributions of qualitative findings as a source of evidence is face validity, including potential participants' acceptability of intervention elements. In our study, the qualitative follow-up feedback obtained from the one child and parent informed parent and child acceptability of our newly developed intervention. Feedback from the one child and one parent was acceptable here as our goal was to prepare and develop the intervention for future standardization and pilot testing in the next study. The child and parent dyad were interviewed separately to "try out" the planned content and format for the newly developed intervention. To assess their acceptability of the digital storytelling format, the child and parent first watched 2 examples of a digital story via YouTube: (a) a video describing the process of digital storytelling (http://www.youtube.com/ watch?v=rUZXBc6yRhU) and (b) a generic example of a digital story about a child's trip to Disney World (http:// w w w. y o u t u b e . c o m / w a t c h ? v = E R C 2 X q c Z P p g ) . Interviewers then asked participants to play a "card-sorting game" to gather their feedback on the guiding questions that would be provided to other kids in the future to create a digital story about themselves. Each question was presented to participants on a 4 × 6 card. Participants were instructed to read each question and sort the cards based on how much they liked or disliked each question (0-do not like [], 1-not sure, 2-like []). At the conclusion of the card-sorting game, questions placed in the "not sure" bin were discussed with each participant to get their feedback on what they liked or disliked about the question. This card-sorting method was based on Lynn's (2011) work in instrument development. Open-ended questions explored parent and child acceptability of the planned intervention.
Although face validity was not claimed due to small sample size, the parent and child's self-reports regarding the planned intervention provided evidence of their acceptability of the intervention, which contributes to face validity. Based on this qualitative work, we further shaped the planned intervention. For example, both the parent and child reported that they did not like the guiding question related to doing chores because that was something children typically do not like to do; therefore, this guiding question was deleted. A few guiding questions were reworded based on the child not understanding the intended meaning. For example, the child did not understand the question, "Who do you look up to and why?" Therefore, we revised the question to "Who is your hero?" This wording was better understood and well received by the child. Qualitative reports also highlighted the need for the intervention to allow for individualization according to each child's preferences. Many guiding questions rated as "do not like" or "not sure" did not apply to the child. For example, the child reported not wanting to talk about her favorite food because she could not eat due to a feeding tube; however, she advised it was a good question to include for other children. In response to this information gathered in our qualitative study, we shaped our intervention to include a worksheet guide for children who would participate in the future planned pilot study. The worksheet included the list of possible guiding questions that children could select as topics to include in their digital story. Questions they did not choose would not be included. We also added the option for children to add topics (eg, their cancer journey) to include in their digital story.
Qualitative data also informed us of child and parent preferences for how much parents would be involved in the intervention. Participants' intervention acceptability self-reports shaped the intervention to allow for child participants to either work alone or seek the help of other family members (eg, parent) in selecting topics, music, or photos for the digital story. The child participant suggested that both parents and children review and provide feedback on drafts of the created digital stories. Therefore, we revised our plan from only showing the digital story drafts to the children to showing the drafts to both the children and their parents. We then added content in our planned parent consent and child assent forms for the future pilot study to inform participants that both children and their parent caregivers would view the digital stories.

Conclusion
As a result of our qualitative approach, we have developed a legacy-making intervention for the pediatric oncology population. Using age-appropriate and sensitive wellthought-through questions, researchers gained a better understanding for what was meaningful to this rare population regarding their legacy-making hopes and desires. Qualitative data resulted in progress toward developing of a conceptual framework for pediatric legacy-making activities, development of an age-appropriate legacy-making intervention, and future pediatric oncology research design components (Table 2 and Figure 1). We have demonstrated how preliminary qualitative data can be used to develop an evidence-based legacy-making intervention that is now ready for pilot testing. Theoretically and empirically grounded pediatric palliative care interventions could contribute to the subsequent well-being of children with cancer and their family members. Developing interventions based on prospective child self-reports can provide empirical evidence for hospital services offered to pediatric cancer patients. The results of our carefully planned qualitative study laid a solid foundation for a newly developed legacymaking intervention in pediatric oncology ready for pilot feasibility and efficacy testing.

Study Components Benefits of Preliminary Qualitative Work
Conceptual/theoretical structure Qualitative data provided the foundation for an empirically based framework for a pediatric legacymaking intervention, informing intervention content and format (Table 1)  Intervention  development Child and parent self-reports informed valid content and structure for a legacy-making intervention Expert appraisal/feedback from parent and child informed necessary revisions to intervention Qualitative data provided evidence for face validity and intervention acceptability Design Revised study language to remain sensitive to individual patient and family needs Assessed feasibility of locating, recruiting, and retaining target population planned for pilot study Recruitment and feasibility data informed revisions to inclusion criteria and study design Environment Informed staff acceptability of supporting project Established staff-researcher relationships necessary for conducting successful subsequent pilot study Pamela S. Hinds, PhD, RN, FAAN, is the director of nursing research at Children's National Medical Center and professor of nursing at George Washington University. An internationally known pediatric oncology palliative care research expert, her work has included qualitative methods to develop substantive theory, behavioral intervention development, and qualitative and quantitative approaches to convey the ill child's experience.