{'fraction_non_alphanumeric': 0.027282924254420693, 'fraction_numerical': 0.005509369226708894, 'mean_word_length': 5.104107933950866, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Patient, Family, Caregiver, and Community Engagement in Research: A Sensibility Evaluation of a Novel Infographic and Planning Guide

Background: Engaging patients, their families, caregivers and the community (PFCCs) throughout the research cycle provides many benets of ensuring that research is meaningful for the population it is meant to benet. Several engagement tools, guidelines, and recommendations have been developed to guide researchers in engaging PFCCs. However, many of these tools are lengthy, complex, and lack the recommended behaviours to facilitate engagement of PFCCs in research. An infographic and planning guide was developed as part of a family engagement in research course including families and researchers in order to facilitate engagement of PFCCs in research. The aim of this study was to evaluate the sensibility of this infographic and accompanying planning. Methods: Thirteen rehabilitation researchers reviewed the PFCC engagement tool, participated in a semi-structured interview, and completed a 10-item sensibility questionnaire. Interviews were transcribed, imported into NVivo Plus, and analyzed using direct content analysis. Median scores and proportions of responses for each of the 10 items in the questionnaire were calculated. Results: Median scores for all questionnaire items were ≥ 4 on a 7-point Likert Scale and none of the items’ median scores were ≤ 3. Participants reported the tool was easy to navigate, contained relevant items to promote PFCC engagement, and followed a logical sequence. Participants’ suggestions for modications included reformatting the tool, redesigning the infographic redesign, and changing the title. Conclusions: The tool was deemed sensible for overt format, purpose and framework, face and content validity, and ease of usage. The tool is useful for researchers who require guidance to engage PFCCs across the research cycle. Further studies are recommended to assess the effectiveness of the tool to engage PFCCs in research. for this part of the tool, such as the infographic’s title and colour contrast. The need for a clear and captivating title is important as it is often the rst text that is encountered by a reader in an infographic. 21 In order to make the title stand out, participants suggested increasing the font size and omitting the use of acronyms. Powerful infographics require enticing color-contrast and appealing visuals to attract a reader’s attention. 22 Thus, the use of white text on grey background in our infographic may have contributed to the infographic’s diminished visual appeal according to participants. With future modications, the infographic has the potential to be more visually appealing and therefore more effective.


Methods
Ethics approval was obtained from the University of Toronto's Health Science Research Ethics Board.

Research Team
This study was conducted by six students (ATG, ML, KC, AG, EG, CR) in the Masters of Science in Physical Therapy program at the University of Toronto, Toronto, Canada. Three advisors supervised the students, two of which (SG, MS) developed the PFCC engagement tool as part of the Kids Brain Health Network, CanChild, and McMaster University's Family Engagement in Research Certi cate of Completion Program taught by the third advisor (AC). The results were interpreted in light of the study team's experiences and expertise.

Study Design
Sensibility was assessed by using a structured questionnaire and semi-structured interview. Development of the questionnaire was based on Feinstein's sensibility framework 13 and a previous questionnaire that was implemented by O'Brien and colleagues to evaluate the sensibility of an HIV questionnaire. 15 The semi-structured interview guide was adapted from Feinstein's sensibility framework in conjunction with guidance from previously conducted sensibility analysis. 13,15 This combination of assessment measures enabled participants the freedom to expand on their views, while also providing structure so that comprehensive information could be obtained.

Recruitment
Since PFCC engagement largely occurs within rehabilitation and health research 14 , rehabilitation researchers were selected as the target population.
Ontario rehabilitation researchers were recruited using snowball and convenience sampling via posters displayed throughout the Rehabilitation Sciences Building at the University of Toronto, email blasts to graduates of the Kids Brain Health Network, CanChild, and McMaster University's Family Engagement in Research Certi cate of Completion Program, a recruitment advertisement in CanChild's newsletter, and social media posts on Twitter and Facebook. Interested participants contacted the research team via email. Participants were screened via phone to determine study eligibility. Participants were included if they resided in Ontario and identi ed themselves as a rehabilitation researcher. 16 Graduate students, post-doctoral fellows, and investigators (early-, mid-, and latecareer) were included to capture diverse perspectives. Participants were excluded if they had previously viewed the PFCC engagement tool or could not speak and read in English. We chose a target sample size of 12-15 participants in order to achieve 'code saturation' (i.e., once no new themes emerge) and 'meaning saturation' (i.e., the point at which identi ed themes can be fully understood). 19

Procedure and Measures
Face-to-face or phone interviews were conducted by two student researchers (EG, ML) who alternated the roles of interviewer and notetaker. Interviews were conducted between February 3, 2020 and March 16, 2020. To ensure privacy and con dentiality, each participant was assigned a pseudonym and data was de-identi ed.

Demographic Questionnaire
Prior to the semi-structured interview, participants completed a demographic questionnaire consisting of age, sex, gender, current stage in career, highest level of education completed, research area, previous experience with PFCC engagement, and previous enrollment in a PFCC engagement course (see supplemental Appendix A).

Sensibility Questionnaire
Prior to the interview, participants were asked to rate their agreement on ten questions related to face validity, content validity, and ease of usage on a sevenpoint Likert scale; a rating of one corresponded to "highly disagree" and a rating of seven corresponded to "highly agree" (see supplemental Appendix B). Survey responses were completed in-person, or verbally scribed for phone interviews and then transcribed by two team members (KC, AG).

Semi-Structured Interviews
The semi-structured interview guide consisted of 14 questions with probes. The questions evaluated each of the ve elements of sensibility identi ed by Feinstein 13 : overt format, purpose and framework, face validity, content validity, and ease of usage (see supplemental Appendix C).

Sensibility Questionnaire
Median scores and proportions of responses for each of the 10 items in the questionnaire were calculated (see supplemental Appendix B). Scores for items ve, six, and seven were reversed prior to median score calculation. Similar to Rowe and Oxman 17 , and O'Brien and colleagues 15 , if the median scores were ≥ 4 for at least 80% of the survey items and none of the items' median scores were ≤ 3, the tool was considered sensible.

Semi-Structured Interviews
The rst interview was manually transcribed by KC and then veri ed by the interviewers (ML, EG) for accuracy. AG and KC then manually transcribed the remaining interviews for a total of six interviews each. Transcribed interviews were imported into NVivo 12 Plus (QSR International Americas Inc, Burlington, MA, USA) on a secure server for data analysis.
Line by line coding of the interviews was performed using direct content analysis. 18 To ensure reliability in coding, ve team members (AC, ATG, SG, CR, MS) coded the rst two transcripts. New themes different from the original coding scheme led to an updated coding structure. The remaining transcripts were coded by two team members (AG, CR) who consulted with one another after the coding of every interview to ensure consistency. Once all the coding was completed, the codes were organized into the most prominent recurring themes by six team members (ATG, AG, KC, EG, ML, CR). Data saturation and code saturation was achieved; no new themes emerged, and a comprehensive understanding of themes occurred.

Participant Demographics
Two of the 15 participants were excluded because they had previously viewed the tool (n = 1) or were not a rehabilitation researcher (n = 1). Thus, 13 Ontariobased rehabilitation researchers were included in the nal sample. Participants were in various stages of their careers, with approximately half (n = 6) in the process of PhD completion. Most participants (n = 8) had previously engaged PFCCs in research, although only one had taken a course on PFCC engagement in research (Table 1).

Sensibility Questionnaire
Results from the sensibility questionnaire can be found in Table 2, where items ve, six, and seven have been reversed scored. The median scores of all items were at least 4, and 9/10 items had a median score of at least 5. Item 6, "there were missing stages in the tool that should be included," did not achieve a median score of 5. The items that scored the highest (i.e., median score of 7) were "order of the stages" and "clarity of the planning guide."

Interviews
Interviews ranged between 17 and 31 minutes long. Findings are described with respect to each sensibility domain (i.e., overt format, purpose and framework, face validity, content validity, and ease of usage) below.

Overall Impression
Most participants (n = 11) had positive overall impressions of the PFCC tool, stating that the tool was well thought-out, and included valuable information to help researchers. Jordan stated, "I think it's very good and I will be using this in the near future." However, two participants who had previous experience with PFCC engagement by virtue of their eld of research had somewhat negative impressions, stating that the tool did not further their knowledge. Mason stated, "because of my background in social sciences, engaging the people that you're doing research about is at the very basis of what you do. So in that sense the tool didn't help me particularly."

Overt Format
Five participants thought the title of the infographic was clear and suitable. Erin stated, "I think it's pretty short and concise and to the point." However, most participants (n = 10), including two who thought the title was clear and suitable, suggested modi cations for the title: removing the PFCC acronym (n = 5), clarifying the infographic's intent to be a step-by-step process intended for researchers (n = 3), and enlarging and capitalizing the title to make it more visible (n = 2).
Most participants (n = 9) held predominantly positive impressions regarding the infographic's overall design, stating it was consistent, aesthetically pleasing, easily readable, used appropriate spacing, and language. Despite this, most participants (n = 11), provided suggestions to improve the PFCC tool. The most commonly reported suggestion (n = 5) pertained to improving the tool's accessibility for individuals with visual impairments by addressing the dark colour contrast and small text size. On the other hand, Mason stated, "I'm actually red-green colour-blind and I could read this, so that's a great sign." Four participants suggested including the infographic earlier in the planning guide to ensure that it is visible to the reader, as echoed by Erin, "somebody may just see references and stop reading." Two participants felt that the infographic was overly crowded. Mason stated, "just overall it's really busy, so maybe if there's a way to reduce some text."

Purpose and Framework
Overall, participants perceived that the purpose of the PFCC tool was to outline possible avenues to engage PFCCs in research, as re ected in Becca's statement, " I think it's really to help from the very beginning, before you even start the engagement process, on how to meaningfully engage with [PFCCs]." Contexts in which participants thought that the tool would be especially helpful included the clinical research environment (n = 4).
Seven participants felt that the PFCC tool could be useful for both PFCCs and researchers, whereas three participants felt that the tool would speci cally bene t researchers more than PFCCs. Five participants said that the PFCC tool would be especially helpful for students or novice researchers, as iterated by Becca who is completing her PhD, "there's a lot of really important information like how to get started...what to do and…[what] I am allowed to do." Four respondents said that the PFCC tool could be more useful for researchers who lack experience with engaging PFCCs in research, as stated by Kai, "it was an excellent guide for those who are new to partnering."

Face Validity
Eight participants found that the tool facilitates PFCC engagement throughout the research process, as re ected by Becca, "it provides a lot of helpful strategies and tips." Five participants felt that the PFCC tool effectively outlined the stages of research that can involve PFCCs, and the associated roles that PFCCs can play.
Certain parts of the tool were found to be confusing and would bene t from editing and clari cation. For example, Gale discussed the clarity of the language, explaining how "with speci c points… [it] was unclear if it's the researcher who's supposed to be doing the [task] or the PFCC." Additionally, ve participants indicated that engaging PFCCs in the research process may be di cult to implement in reality. In particular, ensuring a diverse representation of PFCCs would be challenging, as Jordan stated, "from my supervisor's previous experience, she said that there is a typical...standard. There's certain people that just don't participate and they're having a hard time recruiting a diverse group of people."

Content Validity
Nine participants commented on the stages and checklist items in the planning guide. Six of these participants stated that the stages provided appropriate suggestions on how PFCCs can be incorporated in the research process. Three of these participants said that the checklist items helped break down the stages to provide several options to engage PFCCs in research. Lenny stated, "...The tasks are actually one of the things that I liked best out of the whole information package. It gives a lot of options and a lot of points where you could engage PFCCs." Three participants indicated that they liked that the tool emphasized the involvement of PFCCs across all stages of the research process, speci cally data analysis and interpretation of results. Cam stated, "it's really interesting to think of involving them in troubleshooting in data collection, [and] even looking at the data. They're looking at it with such a different lens. Even just having them help with entering, and analyzing...their interpretation might show things that I would [not] necessarily always see." Nine participants commented that the order of the stages and tasks within the planning guide and infographic were generally appropriate and that they followed a logical ow. Nine participants said they would not make changes to the order, whereas three participants made suggestions to the order of the tasks in the planning guide, Cam stated, "...you have ethics listed here before like funding applications. And it depends on the funding applications, but you often apply for the funding rst."

Ease of Usage
Most participants (n = 8) stated that it took between 15 and 30 minutes to review the PFCC tool; however, it took three participants less than 15 minutes, and two participants more than 30 minutes. Five participants stated that a more detailed analysis of the tool would require more time. Participants suggested that reviewing any PFCC tool would ideally take between 10-15 minutes (n = 4), followed by 20-30 minutes (n = 3). Three participants suggested that the tool should be shorter in order to motivate busy, tenured researchers to take the time to review it. Erin stated, "…but I can see it being an issue…for people who are higher up in research… I can see them not really wanting to open up a 16 page document and read all the way through it." Eleven participants thought that the tool was easy to understand and required an appropriate amount of effort to go through. Of these participants, six reported that formatting details (i.e., table of contents, graphics/charts, lists) were helpful for breaking up sections of heavy text, "which made it easier to read through" (Harper). In contrast, four participants found the planning guide confusing, stating that it was over-complicated, failed to provide context, and would bene t from more visuals. Six participants thought that the planning guide complimented the infographic well. Erin stated that the infographic provided, "a nice, short, and concise way of outlining everything." However, three participants felt that there was a missing link between the planning guide and the infographic. Mason stated, "I know that the planning guide informed the infographic but when I looked at the infographic I didn't think about the planning guide."

Recommendations for Modi cation
A variety of recommendations/modi cations/additions to the PFCC tool were provided which spanned content validity, face validity, and ease of usage. Table   3 provides a summary of the proposed modi cation.

Discussion
Similar to other existing engagement tools that guide PFCC engagement, the sensibility of our tool has not been evaluated, inspiring the current study. Our sensibility evaluation, based on Feinstein's framework, 1 indicated that our tool is sensible in helping guide researchers to engage PFCCs in the research process. Participants valued the provision of concrete, actionable steps in the PFCC tool. However, interview ndings indicated that prior experience with PFCC engagement in uenced how participants perceived our engagement tool. Furthermore, although many participants felt that the infographic and planning guide represented an effective way to share information, concrete recommendations were provided to improve the tool. These included the provision of further exemplars and consideration of including underrepresented groups. Finally, participants expressed their appreciation of the multiplicity of roles that PFCCs can have in research beyond recruitment. Each of these concepts will be discussed below.

The PFCC Tool Provides Actionable Behaviours to Engage PFCCs in Research
Including PFCCs in research involves a deliberate plan, as it is an evolving process that takes time and effort by both researchers and PFCCs. However, lack of knowledge and time constraints to involve PFCCs throughout the research cycle are common concerns among researchers. 6 To bridge this gap, participants felt that the tool's purpose was to outline possible avenues and practical steps to engage PFCCs in research. Participants believed the PFCC tool appropriately highlighted the main research tasks that can involve PFCCs including providing a detailed description of what PFCC engagement is, which roles PFCCs may hold, and how to best involve them. The guidance and knowledge this tool provides may help to address the barriers researchers face towards successful PFCC engagement.

Prior Experience with PFCC is Relevant
Differences in perceptions of the tool may be attributed to the participants' varying experience with PFCC engagement. Health researchers with little to no experience with PFCC engagement have attributed their lack of PFCC engagement to a poor understanding of what constitutes PFCC engagement in conjunction with limited guidance from their institutions. 19 The barriers faced by researchers who are inexperienced with PFCC engagement aligns with our ndings as participants with minimal PFCC engagement experience wished for a more comprehensive tool with additional resources in their eld of research. Conversely, those with extensive PFCC engagement experience believed the tool did not broaden their existing knowledge. This is expected as those with extensive experience are familiar with the concept of engaging PFCCs. 19 Researchers with extensive PFCC experience cite limitations to engagement as lack of funding and compensation for PFCCs, rather than lack of knowledge of how to engage PFCCs in research. To highlight this point, Nilsen et al. 20 noted that consumer involvement in health research often requires greater physical resources, possibly resulting in signi cant resistance from institutions and grantawarding bodies. Overall, the difference in self-e cacy between those experienced and inexperienced in PFCC engagement suggests the need to consider PFCC engagement experience when developing resources and tools that guide PFCC engagement. Researchers with little or no experience may bene t from our tool more than experienced researchers since it aims to provide background information about PFCC engagement and possible avenues to engage them throughout the research process.

Infographics are Effective Mediums to Share Information
In order for infographics to be effective, they should have a compelling title, use illustrations over text where possible, be visually appealing, and follow a clear narrative. 21,22 Our infographic was designed to deliver the necessary information and guidance on PFCC engagement while overcoming the associated time barriers associated with learning about PFCC engagement. Although the majority of participants resonated with the infographic because it gave them a visual aid to quickly refer to throughout the research process, many participants suggested modi cations for this part of the tool, such as the infographic's title and colour contrast. The need for a clear and captivating title is important as it is often the rst text that is encountered by a reader in an infographic. 21 In order to make the title stand out, participants suggested increasing the font size and omitting the use of acronyms. Powerful infographics require enticing colorcontrast and appealing visuals to attract a reader's attention. 22 Thus, the use of white text on grey background in our infographic may have contributed to the infographic's diminished visual appeal according to participants. With future modi cations, the infographic has the potential to be more visually appealing and therefore more effective.

Ensure Underrepresented Groups are Included
Historically, minority groups have been underrepresented in research, 23,24 resulting in decreased applicability of research ndings to these groups. 23 Participants highlighted that the perspectives of minority groups were lacking from the PFCC tool and engaging minority groups is di cult among researchers. This is not unexpected as other engagement tools lack suggestions on how to engage minority groups. 10,25 Engaging underrepresented groups in research ensures that diverse populations are equally depicted. Minority groups can provide distinctive perspectives for developing research topics, assisting with data analysis, and participating in knowledge translation to ensure meaningful applicability of the research ndings. 23 Participants speci cally suggested that the PFCC tool should incorporate targeted strategies for recruitment and engagement of individuals of minority populations.

PFCCs Can Engage in Research Beyond Recruitment
It is important to include PFCCs throughout the research process due to the multitude of proposed bene ts, including increased participant recruitment and improved dissemination of ndings. 6,14 However, there is a lack of PFCC involvement in later stages of the research process, with the majority of PFCC engagement occurring during study design and recruitment. 6 Our PFCC engagement tool provided suggestions for researchers to engage PFCCs in the earlier stages of research with continued engagement with each subsequent stage. Speci cally, one role deemed extremely valuable by participants was the involvement of PFCCs in data analysis, as many participants had not previously considered this option. The insightful contributions of PFCCs in data analysis can ensure that the research ndings are adequately re ected and translated for practical use by the community. 26 For PFCC engagement to occur successfully throughout the entire research cycle, recommendations include engaging PFCCs as early as possible, de ning the roles and duties of PFCCs and the researcher, and maintaining PFCCs' involvement until the research process is complete. 14 The PFCC tool accomplishes this by clearly depicting several methods that PFCCs can be engaged within each respective stage of the research process. This is substantiated by participants as the majority commented that it is bene cial to have speci c actionable steps to engage PFCCs throughout the research process.

Exemplars are Helpful
Involvement of PFCCs in research can provide several challenges for researchers. 6 Participants expressed uncertainty regarding the active role that PFCCs can have in the research process, how to vet PFCCs, and the means of ensuring meaningful PFCC contributions. Although the tool provided appropriate tasks to guide researchers to include PFCCs, the tool lacked su cient concrete examples of how to effectively complete these tasks. Participants requested the addition of pragmatic examples of successful PFCC engagement to better understand how to implement these in their own research. Future addition of examples could provide users with guidance on how to overcome potential challenges and highlight the bene ts of PFCC engagement in research.

Limitations and Future Directions
The strengths and limitations of our study must be taken into consideration when interpreting our ndings. Although we used a semi-structured interview guide, two members of the research team conducted the interviews. Individual interviewer styles could have led to variation in questions and probes, eliciting varied responses from participants.
Insu cient direction was provided to participants with regards to reviewing the PFCC tool prior to the administration of the questionnaire and interview. For example, some participants had failed to review the infographic prior to the interview because they did not read the content in the appendix of the planning guide. Consequently, this impacted their responses to certain questions.
Although an inclusion criterion was that participants had to be Ontario-based rehabilitation researchers, most participants were from the Greater Toronto Area. Therefore, our results may not accurately represent the views and experiences of researchers across Ontario or across Canada. Likewise, our cohort of rehabilitation researchers limits the generalizability of our ndings. Our sample had an unequal representation of male (n = 2) versus female (n = 11) participants, which may not represent potential differences in the challenges faced by males and females with PFCC engagement. Our sensibility questionnaire was not a validated measure; however, it was based on previous sensibility questionnaires that used Feinstein's sensibility framework. 13,15,17 Future directions involve incorporating participant recommendations into revisions of the tool, investigating the e cacy of the PFCC tool on engaging PFCCS in research, and reassessing its sensibility with a larger and further diversi ed sample.

Conclusion
In conclusion, the PFCC tool is sensible with respect to overt format, purpose and framework, face validity, content validity, and ease of usage. The results from our study outline areas for improvement with future revisions of our tool. This includes changing the design of the infographic, re-wording the title, content changes such as taking minority groups into consideration, and alterations to the tool's language to clarify the roles of the researcher and PFCC. Our PFCC engagement tool can be readily used by beginner or experienced researchers who require guidance on how to engage PFCCs across the entirety of the research process.   Note. *items reverse-scored prior to inputting into the table. "...include some resources and some mention that engaging in different groups might require different kinds of skills or different ways of working with those groups." -Kai Adding pragmatic examples throughout the planning guide (n = 4) "..There needs to be a box of real-life scenarios in the appendices. This would give researchers an idea of what you do in a certain situation and this is how it was successfully handled" -Kai Addition of information to guide PFCC recruitment (n = 3) "... vetting, interviews, CVs, does any of that come into play? -Finn Moving the funding application step before ethics (n = 3) "You have ethics listed here before like funding applications. It depends on the funding applications, but you often apply for the funding rst" -Iggy Indicating that the research process is cyclical and not a linear process (n = 3) "Going back and forth between stages is something that is a good thing. It should be encouraged to do that or at least consider that these stages don't often happen in a sequential order." -Mason Acknowledge the power imbalance between the PFCC and researcher/stakeholder (n = 2) "...chang[e] the language and...more explicitly acknowledg[e] that there will always be a power differential between the researcher and the PFCC." -Lenny

Ease of Usage
Modi cation of hyperlinks within planning guide (n = 4) i) directly embed information from external resources ii) modify links to make them clickable within pdf "...it would be useful if some of that information can be included in the tool itself. Just so you wouldn't have to go to so many different websites…, it would be also nice if you can click the links. Because I think you have to type them out." -Jordan Modi cations to the planning guide and infographic to promote better consistency/unity between them (n = 3) "... it's important that the headings are the same headings that are used in the guide so you can look at each, and not wonder is that supposed to be related to that? " -Kai