{'fraction_non_alphanumeric': 0.022324422726837215, 'fraction_numerical': 0.004950336942288653, 'mean_word_length': 5.1710681907765075, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Operationalizing a patient engagement plan for health research: Sharing a codesigned planning template from a national clinical trial

Abstract Introduction Engaging with patients about their lived experience of health and illness and their experience within the healthcare system can help inform the provision of care, health policies and health research. In the context of health research, however, operationalizing the levels of patient engagement is not straightforward. We suggest that a key challenge to the routine inclusion of patients as partners in health research is a lack of tangible guidance regarding how this can be accomplished. Methods In this article, we provide guidance on how to codesign and operationalize a concrete patient engagement plan for any health research project. Results We illustrate a seven‐step approach using the example of a national clinical trial in Canada and provide a patient engagement planning template for use in any health research project. Conclusion Such concrete guidance should improve the design and reporting of patient engagement in health research. Patient or Public Contribution The De‐Implementing Wisely Research group is informed by a national 9‐member patient partner council (PPC). The research team includes three lead patient partners who are coinvestigators on the grant that funds the program of research. Members of the council advise on all aspects of the study design and implementation. The ideas presented in this paper were informed by regular communication and planning with the PPC; specific contributions of lead patient partner authors are outlined as follows: Brian Johnston, Susan Goold and Vanessa Francis are patient partners with a wide breadth of experience in the healthcare system and health research projects. The guidance in this article draws on their lived and professional expertise. All patient partner authors contributed to the planning of the manuscript, participated in meetings to develop content and provided critical manuscript edits and comments on drafts.

In this article, we provide guidance on how to codesign and operationalize a detailed PEP for any health research project. We illustrate a seven-step approach using the example of a national clinical trial in Canada and offer lessons learned in the process. We propose this guide as a resource for research funders, researchers, providers and patient partners with the goal of advancing a more rigorous approach to the development and implementation of patient engagement in health research.

| A growing consensus: Patient engagement in health research
Multiple frameworks, best practices, checklists and questions to guide research teams now exist for patient engagement, [1][2][3][4][5][6][7] driven in part by the call from leading funders and journals to partner with patients in the conduct of health research. 6,[8][9][10] Evidence suggests that partnering with patients in the design and conduct of health research can improve both research quality and outcomes. [11][12][13] For example, patient engagement helps ensure that research questions and study measures reflect patient-identified priorities. 11 Engagement has helped increase study recruitment and contributed to the design of informed consent documents and other study materials, as well as helped the translation of research findings into practice. [11][12][13][14] Engagement occurs across all stages of the research process from conceptualisation of study questions and protocols, to defining and choosing study measures and outcomes, to helping to collect or analyse study data, to knowledge translation and dissemination efforts. 5,8,11,12,14 1.2 | Levels of involvement Patient engagement is often described on a continuum from lower levels of ability to influence decisions to higher levels that provide decision-making authority. 5,15 The International Association for Public Participation (IAP2) identifies five levels of engagement ranging from lesser to greater levels of engagement: inform, consult, involve, collaborate and empower. 15 Conceptually, as one moves up the spectrum of engagement, greater levels of decision-making power are available to patient partners. At the level of consult, for example, patients may be asked for their opinions about a project, but the use of their input is not guaranteed. At the level of collaboration, however, researchers commit to working with patient partners in each aspect of research decisions, including formulating decision options (e.g., what methods to use) and preferred solutions. 15 It is becoming a well-recognized and accepted standard for describing public involvement, 5,6 and has been research-modified ( Figure 1) to outline these levels and what they mean for engagement with stakeholders in research projects. 16 It is a useful framework for thinking through how patients can be engaged at different levels for all health research project activities as we outline below.
While there are some exceptions, 17 patient engagement in health research generally remains limited to preliminary activities or early stages of a project (e.g., identifying research priorities, helping with study recruitment), rather than later stages in the research lifecycle, such as data collection and analysis. 1,5,11,13,14 This is largely related to barriers such as resource constraints (e.g., time, funding), but also limited availability or awareness of guiding frameworks and Health Research (CIHR) oversees the SPOR, which is focused on ensuring that the right patient receives the right intervention at the right time and is committed to partnering with patients in the design and conduct of research. 10 In the SPOR, patient engagement is defined as 'meaningful and active collaboration in governance, priority setting, conducting research and knowledge translation'. Patients who actively contribute to research are defined as 'patient partners', including not just individuals with lived experience of a health condition, but also informal caregivers, including family and friends. 10 Low-value care, healthcare with little to no clinical benefit or that causes harm-is a challenge for healthcare systems globally, including Canada, resulting in direct harm to patients (due to adverse effects of treatments or secondary unwarranted tests) and threatening healthcare system sustainability. 18,19 Choosing Wisely Canada (CWC) is a national campaign engaging healthcare professionals and patients to identify low-value care. CWC has developed a patient and public engagement framework and engaged in a wide range of activities to support patients and citizen involvement in CWC and increase awareness of the problem of low-value care. 20,21 There remains considerable uncertainty about how best to deimplement low-value care, suggesting the need for implementation research in this area. This project (see protocol in Grimshaw et al. 22  Two trials in each province will evaluate theoretically informed deimplementation strategies, targeting both healthcare professionals and patients to reduce two low-value care practices (preoperative testing in low-risk ambulatory surgery and imaging in uncomplicated low-back pain). Here, the PEP for the study site in Newfoundland and Labrador (NL), Canada, is presented for 1 year of the low-back pain trial. Once this initial relationship building and work had occurred, however, both researchers and patient partners acknowledged that uncertainty remained about how best to enact specific patient engagement activities for each trial, in each province. While the council's ToR was detailed and comprehensive, it remained challenging for the team to be concrete and explicit about specific roles and tasks.

| Getting down to the specifics: Codesigning the PEP
Each provincial research team will work with their local patient partners to codesign a PEP for the two trials to be carried out in their province. The NL team took the lead on this task for the subproject on reducing imaging for nonspecific low-back pain because of the patient engagement expertise of the local team. The AB and ON teams will replicate the process.
This approach was chosen after many months of struggling to articulate a complex patient engagement strategy for a multisite, multiyear research project. The team was challenged by varying research timelines across the three study sites. For example, the NL site had completed some data collection, while other sites were in the process of submitting ethics applications. The COVID-19 pandemic further challenged timelines in the three provinces. Ultimately, the approach taken by the team was to develop a PEP for each trial area (preoperative testing in low-risk ambulatory surgery and imaging in uncomplicated low-back pain), for each province, and to do so for 1 year at a time. This approach allowed a clearer understanding of what project activities would realistically be undertaken in any given year and focused clearly on the activities related to one trial behaviour. Ultimately, this proved a much more feasible approach that not only outlined specific project activities for 1 year but also provided patient partners flexibility to be involved in the activities of interest to them, at a level with which they were comfortable.
This also ensures that the PEP is reviewed at least once annually, the minimum we recommend (and more often in the event of unanticipated project changes).

| RESULTS
Below, a seven-step approach for engaging patients in research and codesigning a PEP is presented. A fillable PEP template is provided ( Figure 2), while a fully worked example from 1 year of our trial is provided in Table 1. Note that in Table 1, we have used the position of the team member to indicate which team member is responsible for each activity and we indicate researcher time by initials. In cocreating PEPs within teams, however, we recommend identifying specific team members responsible for activities by name.
It is important to note that references to the 'research team', acknowledge patients as full partners on the team. The language of 'researchers' and 'patient partners' is not meant to imply a 'we' versus 'them' attitude; rather, it is used to clearly delineate the roles and responsibilities of all team members. Note also, that while a seven-step approach is presented, the development of the PEP was not linear. It involved multiple iterations amongst patient partners and researchers, with several drafts specifically informed by patient partners' requests (e.g., information on approximate time commitment or whether training was necessary for a given activity). Throughout the year-long development of the PEP, the process was presented at multiple national patient partner council meetings and national research team meetings so team members from other provinces could learn from NL efforts. The plan has been well received, and a second site (AB) recently used the template to create its specific provincial PEP. Patient partners have also shared the plan with other research teams with which they work. It was this transferability that led to our decision to publish the process and plan so other research teams could benefit. step, as this helps shape the approach to patient engagement, clarify its goals and the scope of patients' involvement in the research project. The IAP2 Spectrum of Engagement modified for research 16 was chosen for this study, as it clearly identified levels of patient involvement ( Figure 1) and what the promise as researchers would be to patient partners. It also provided a clear degree of choice to patient partners in how they wished to be involved in specific project activities. This was particularly important, not only because the choice of involvement is the best practice for patient engagement 3,5,8 but also because patient partners and researchers struggled with the exact nature of engagement for any given project activity. Using the levels of involvement in the research-modified IAP2 spectrum helped all team members critically reflect on what levels of involvement were possible for each project activity and what the responsibility of each team member would be at any given level.

| A seven-step approach
There are a variety of tools and frameworks in the literature to help plan patient engagement activities in research. 1,2,5,23 Any of these can be useful guides for research teams, particularly those new to patient engagement as all provide a systematic way of considering the myriad choices to be made in engaging with patient partners in research. However, to our knowledge, there is no consensus in the literature on the 'best' tool or framework to use.
The steps that follow remain relevant to any framework that explicitly includes consideration of the levels of patient engagement. 5,23 For our purposes, we wanted to be as concrete as possible about each and every project activity that could benefit from patient This project activity includes helping to translate important messages about why we are trying to change practice and the key barriers to following low-back pain guidelines that were found in our assessment. We want to translate the results from the barriers assessment into understandable focused messages that are relevant for the different knowledge users. The knowledge users for the study will be researchers, clinicians, patients and policymakers. Levels of patient engagement for this activity are outlined below. We will keep you informed by regularly communicating with you and providing you with copies of materials within 1-2 weeks of producing them.

Inform
• The patient engagement coordinator will email you copies of materials we produce • Review drafts of the key messages and dissemination materials crafted for each user group.

Consult
• Provide written feedback (emailed to Research Manager NL) on the accessibility/usability of the material from your perspective within 1-2 weeks of receiving these documents.

Researcher commitment
We will consider your feedback as we prepare the final versions of key messages and dissemination materials and let you know how your feedback influenced the process.
• Research Manager NL will email you draft copies of the materials we have produced (cc PE coordinator).
• PE coordinator will communicate to you how your input has influenced our work.

Meetings required Patient time
Researcher time Training required Responsible team member • Review drafts of the key messages and dissemination materials crafted for each user group.
• Provide written feedback on the accessibility/usability of the material from your perspective and help to generate a list of potential frequently asked questions within 1-2 weeks of receiving these documents.

Researcher commitment
We will work with you to ensure your feedback is incorporated and directly reflected in the finished product to the maximum extent possible (while maintaining the integrity of the research).
(Continues) • Attend a lecture on the TDF.
• Partner with researchers on the team to translate the key messages and identify the most appropriate communication/dissemination methods. This includes: o participating in training on the theoretical domains framework, o attending meetings with selected team members to review the results, cocreate key messages based on those results and write the text for a 1-page infographic report, o reviewing and commenting on subsequent drafts.
• Become a local champion of the key message by spreading the key message at different functions and through relevant personal networks if possible.

Researcher commitment
We will work with you to cocreate dissemination materials and select communication strategies, relying on your advice and innovation in this area. We will incorporate your advice and recommendations to the maximum extent possible.
• Research Managers and a local facilitator with meet with you to review the results of the barriers assessment and begin work on breaking the results down into key messages. • Subsequent meetings will focus on developing the text for a 1-page infographic report, selecting communication channels, and reviewing edits and feedback on various drafts of the report.

September-December 2020
Includes reviewing the problems identified in the barriers assessment, determining which problems were most important and able to be targeted and matching them with appropriate behaviour change techniques.

Researcher commitment
We will keep you informed by telling you when the behaviour change techniques (BCTs) have been selected by the stakeholder committee.
• PE coordinator will email you correspondence within 1-2 weeks of making these decisions, indicating which behaviour change techniques have been selected to form our intervention and what barriers they address • Attend a brief foundational lecture on BCTs.
• Review a summary of BCTs selected to target the problems identified in the barriers assessment and the process for delivering the intervention.
• Provide written feedback (emailed to Research Manager NL) highlighting your concerns or questions you have with the approach we have selected within 1-2 weeks of receiving the summary.

Researcher commitment
We will consider your feedback and let you know how it influenced the intervention development.
• Senior Research Manager, National site will deliver a short foundational lecture for patients.
• Research Manager NL will email you a summary of the BCTs selected to target relevant barriers identified in the barriers assessment and the process for delivering the intervention (cc PE coordinator). • After we have reviewed your feedback, the PE coordinator will communicate to you how your participation influenced intervention development. • Review a summary of BCTs selected to target the problems identified in the barriers assessment and the process for delivering the intervention.
• Provide written feedback (emailed to Research Manager) highlighting your concerns or questions you have with the approach we have selected within 1-2 weeks of receiving the summary. • Attend a round-table meeting to review your critique and determine how the team can incorporate your feedback.

Researcher commitment
We will work with you to ensure your feedback is incorporated and directly reflected in the intervention that is developed to the maximum extent possible.
• Senior Research Manager, National site will deliver a short foundational lecture for patients.
• Research Manager, NL will email you draft copies of the materials we have produced (cc PE coordinator).
• Selected members of the NL team will meet with you to review your feedback and discuss how it can be incorporated. • Participate in focused conversations, team meetings, and larger stakeholder group meetings to review results of barriers assessment and potential BCTs for this intervention o help to select appropriate BCTs to target the problems identified in the barriers assessment and o plan the process for delivering the intervention.
• Help to find participants (e.g., patients or others depending on your individual networks) for a larger, key stakeholder group who will be advising on the intervention development and materials moving forward.

Researcher commitment
We will work with you to codevelop the intervention for the LBP project, relying on your advice and unique perspective. We will incorporate your advice and recommendations to the maximum extent possible.
• PE coordinator will advise you on where to find the online BCT training.
• The Senior Research Manager, national site and the PE coordinator with meet with you to review the training and answer any questions you may have. • Subsequent meetings with the NL team and the larger group of key stakeholders will focus on selecting the BCTs and determining how the intervention will be delivered.

October-December 2020
Includes developing physician and patient education materials that will be used in intervention delivery (e.g., anything developed for physicians or patients (videos, pamphlets, etc.) Patient responsibilities • Review the intervention materials that are selected for the intervention within 1-2 weeks of receiving this documentation.

Researcher commitment
We will keep you informed by sharing the intervention materials we have selected with you.
• PE coordinator will email you the intervention materials that have been developed/selected for this intervention. • Review the intervention materials produced and/or selected for the intervention (may include videos or scripts for family doctors, education or treatment planning materials for patients).

Researcher commitment
We will consider your feedback and let you know how it influenced the intervention development.
• Research Manager, NL will email you the intervention materials (cc PE coordinator).
• After we have reviewed your feedback the PE coordinator will communicate to you how your participation influenced our intervention materials.

Researcher commitment
We will work with you to ensure your feedback is incorporated and directly reflected in the intervention that is developed to the maximum extent possible.
• Research Manager NL will email you draft copies of the materials we have produced (cc PE coordinator) • Selected members of the NL team will meet with you to review your feedback and discuss how it can be incorporated. Other project activities may also be outlined in the Gannt chart (e.g., project administration) that are not included in our PEP. Regular and ongoing discussion with our patient partners during council meetings revealed their general lack of interest in these activities; as such, they are not included in our final PEP. However, at this stage, patient partners can suggest other activities that come to mind that may not be included in the original project plan. In our project, for example, patient partners acknowledged that they did not want to be involved in weekly operational meetings where site leads discussed new hires, new students or human resource issues. However, in discussion with site leads, they asked to be kept informed of news arising from operational meetings; as such, the team agreed to a weekly email of bulleted points summarizing these discussions.

| Step 3: Select potential levels of patient engagement for each project activity requiring patient engagement
Once the team decides on the activities of the project that require/ would benefit most from PE, careful thought is needed about at what level engagement can be offered, for each project activity. At this stage, select as many levels of engagement as possible to ensure opportunities for higher levels of engagement are not lost. At this point, it is important to address any feasibility considerations. The team must consider time, resources, supports or training required for patients and researchers at each level of the engagement plan, and who will coordinate this study. Higher levels of engagement will necessitate more resources and a greater need for facilitation and general coordination (see Table 1  • Attend a lecture on the TDF. Note: This is the same training required at the collaboration level for the barriers assessment. If you have completed the lecture for the barriers assessment, you are not required to repeat it. • Partner with researchers and other stakeholders to select and potentially modify or develop intervention materials. This includes: o attending meetings with selected team members to review intervention materials relevant to the BCTs selected during intervention development, o Cocreate new materials and/or modify existing materials (as required), o review and comment on materials selected.
• Help to find participants (e.g., patients or others depending on your individual networks) for a larger, key stakeholder group who will be advising on the intervention development and materials moving forward.

Researcher commitment
We will work with you to select, modify, and/or create intervention materials to support the implementation of the selected BCTs, relying on your advice and unique perspective. We will incorporate your advice and recommendations to the maximum extent possible.
• The PE coordinator will send you copies of any relevant materials that the team is already aware of for your review before meeting.
• Subsequent meetings with the NL team and a larger group of key stakeholders will focus on selecting the intervention materials, identifying how they should be modified, and creating new materials as needed. • The Scientific PE Lead will facilitate these meetings. ETCHEGARY ET AL. | 707 resource needs for each level of engagement, for each project activity. In Canada, we recommend engaging with provincial SUPPORT Units 10 whose role is to support and build capacity for patient en-  (Table 1).
It is clear how greater levels of engagement require more time commitments from all team members (Table 1). At the levels of inform and consult, patient partners are asked mainly to review materials and provide written feedback at the consult level. However, at the collaborate level, patient partners are asked to help design materials, attend meetings where intervention materials will be reviewed, and attend a training session specific to the theoretical framework that guides intervention development. The researcher commitments and time required are also greater at this level.

| A template for others to follow
Below, a fillable PEP template is presented (Figure 2). It highlights the kind of information that should be crafted for each section of the plan, regardless of the research project. It provides a guide for others to follow and should encourage critical and deep reflection on specific project activities, possible levels of engagement for each activity and resources available to enact engagement.

| Evaluating the plan
The evaluation of patient engagement remains understudied, making it difficult to know what works best and when. 5,13,14 The research team will evaluate the use of our PEP for year one before finalizing plans for subsequent project years. While the COVID-19 pandemic has slowed or halted data collection in all three study sites, an unforeseen benefit has been the luxury of time to create a detailed PEP and its evaluation. The team anticipates a mixed-method participatory evaluation, which some team members have already used successfully in evaluating patient-oriented research. 14 We anticipate data collection through the use of online surveys and interviews, as well as in the collection of administrative, descriptive data (e.g., were follow up emails sent to patient partners as outlined in the plan). Key evaluation questions will potentially include: Did all team members meet all commitments as outlined in the plan?
Was any training provided sufficient to enable team members to engage at the levels they preferred?
Were there any setbacks/lessons learned? What could be improved about patient engagement activities?
What was the patient and researcher experience of enacting the plan? What could we do differently?
Did other project sites use our template? Did patient partners share the plan with other project teams with whom they work?

| Conclusions and lessons learned
Engaging with patient partners in health research is now largely accepted, and growing evidence supports its ability to improve research relevance, quality and outcomes. 11,12,14 Despite a growing number of tools and frameworks, however, there remains relatively little tangible guidance for how to operationalize the levels of possible engagement across health research activities. The detailed PEP provided here is a practical template that can be used widely by other health researchers and patient partners, customized to fit any specific project. We conclude with key lessons learned in the process of creating the plan.
First, Anticipate and budget for costs-The successful creation and implementation of a PEP requires targeted resources. In particular, a dedicated research staff person who coordinates the process is desired; budget for such research support at the funding application stage. Second, Be transparent-It is critical to identify those areas where patient engagement will not be practically possible in a project before creating the plan (e.g., members of our patient partner council who were recruited after funding was obtained had no opportunity to advise on study research questions or the theoretical framework that guides the study as these decisions were already made). Also identify if a particular level of engagement is not possible for all specific project activities (e.g., empowerment was not available for the project activities of staff hiring or financial management and reporting, nor for most research activities). Third, recognize that Specifics matter ETCHEGARY ET AL. | 709 -Patient preferences can change when they are given additional information. For example, all three of our patient partners had initially chosen the collaborate level of involvement for many project activities in earlier verbal discussions and in earlier (less detailed) versions of the PEP. Through iterative drafts and reviews of the template, however, and at patient partners' request, we added additional information about training and time commitments. When this more precise information was included, our patient partners chose the consult or involve level of engagement instead. We believe this level of detail is important if we are to foster the levels of engagement patients truly wish. Cocreate-While we highly endorse cocreating PEPs with patient partners, we recommend researchers and research staff create the initial draft. These are the team members who best know specific project details and timelines, both of which are crucial to formulating a PEP. Once that initial draft is shared, however, patient partners are the team members who will then best be able to identify missing information gaps, and their feedback must drive all further iterations. Finally, Be flexible-If the COVID-19 pandemic has taught us anything, it is to be flexible. In this project, the pandemic did alter data collection activities, particularly in the Ontario and Alberta sites. It is important to discuss with patient partners whether alternative activities are available or update timelines regularly. Our council has continued to meet regularly throughout the pandemic, which has been important to maintain trust, communication and motivation. It also provides the opportunity for social support and brainstorming new ideas (e.g., creating potential knowledge translation outputs related to patient engagement, while waiting for data collection activities to resume in full).
In this paper, we aimed to provide guidance on how to codesign and operationalize a detailed PEP for any health research project. We not only include a fully worked example of a PEP using our trial as an example but also provide a fillable template for others to use in any health research project. We hope this guide can serve as a resource for research funders, researchers, providers and patient partners with the goal of advancing a more rigorous approach to the development and implementation of patient engagement in health research.