{'fraction_non_alphanumeric': 0.015451947123972848, 'fraction_numerical': 0.0008931761343336906, 'mean_word_length': 5.38369441277081, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Unpacking the potential of developmental evaluation in codesign work

Over the past decade, there have been rapid developments and growing commitments to codesign as a tool for patient and public engagement in health research and service improvement, particularly with populations who face social and systemic barriers to accessing services. In fact, the uptake of codesign has contributed to what has been called a ‘Participatory Zeitgeist’, whereby codesign and coproduction have become the spirit of our contemporary times. Despite growing attention and uptake of codesign approaches and the potential for positive impact, there continue to be significant gaps and inconsistencies in evaluation. Health Expectations has published much of this study, including a recent systematic review of evaluation in patient and public engagement in research and health system decision‐making. While the review found a growing number of published evaluation tools, it also found that many of these tools lacked an explicit conceptual framework that is needed to link empirical evaluation with a theoretical foundation. In this editorial, we address this need for theoretically informed, rigorous, and systematic evaluation of codesign through a critical reflection of our own experiences. Without a strong theoretical foundation or critical lens, codesign can reproduce existing hierarchies. We argue for the value of a developmental evaluation (DE) approach as an alternative to traditional evaluation methods, and one that is compatible with the philosophy and principles of codesign that embraces collaboration, complexity, reflexivity and emergent outcomes.


Unpacking the potential of developmental evaluation in codesign work
Over the past decade, there have been rapid developments and growing commitments to codesign as a tool for patient and public engagement in health research and service improvement, particularly with populations who face social and systemic barriers to accessing services. In fact, the uptake of codesign has contributed to what has been called a 'Participatory Zeitgeist', whereby codesign and coproduction have become the spirit of our contemporary times. 1 Despite growing attention and uptake of codesign approaches and the potential for positive impact, there continue to be significant gaps and inconsistencies in evaluation.
Health Expectations has published much of this study, including a recent systematic review of evaluation in patient and public engagement in research and health system decision-making. 2 While the review found a growing number of published evaluation tools, it also found that many of these tools lacked an explicit conceptual framework that is needed to link empirical evaluation with a theoretical foundation.
In this editorial, we address this need for theoretically informed, rigorous, and systematic evaluation of codesign through a critical reflection of our own experiences. Without a strong theoretical foundation or critical lens, codesign can reproduce existing hierarchies. We argue for the value of a developmental evaluation (DE) approach as an alternative to traditional evaluation methods, and one that is compatible with the philosophy and principles of codesign that embraces collaboration, complexity, reflexivity and emergent outcomes.

WHY DE?
There are a range of approaches that can be used to evaluate codesign work from simple pre/posttest evaluations to randomized control trials with embedded process evaluations. 1 These approaches are based on traditional assumptions about researcher objectivity, and typically measure performance and success against predetermined goals and outcomes. Most of the time, however, codesign is complex and evolving and it may be difficult to predetermine activities and outcomes. This is especially challenging when the process calls for active engagement of stakeholders, such as service providers and service users including those from equity-deserving groups, in

WHAT IS DE AND HOW DO YOU DO IT?
DE is an approach to evaluation that emphasizes innovation and learning. It was developed by Michael Quinn Patton and colleagues in response to the challenges of understanding complexity in human systems and the need to provide structured and useful information for decision-making in a way that supports innovation. 4 DE is grounded in pragmatism, and systems and complexity theories.
Pragmatism is based on the premise of using the most appropriate sources of data and knowledge to investigate real-world system-level problems where complex social issues need multipronged approaches. This theoretical foundation aligns well with patient-oriented research by prioritizing democratic values and collaborative approaches to understand and improve a system by identifying underlying power structures and how different components collectively contribute to outcomes. 5 Our use of DE was driven by challenges in specifying The Hub's activities and outcomes in advance of implementation. The Hub's initial goals were quite broad ('to engage, educate and innovate'), and we wanted to engage key stakeholders in identifying priorities and approaches to implementation to ensure that we were responsive to the needs of our target audience (i.e., equity-seeking communities, trainees, service users, service providers, researchers, policy makers).
We hired an external evaluator who observed team meetings, re- We used arts-based and storytelling methods to visualize stakeholder experiences and perspectives in the cocreation of the Hub.
For example, focus group café participants were asked to bring an object to depict their experience with codesign; this was shared at the outset of the meeting to spark conversations about the opportunities and challenges of codesign. Our data is available from the corresponding author upon reasonable request.
As we developed our evaluation strategy, we struggled to define vulnerability and community in a way that would focus on structural oppression rather than individual challenges and identify dimensions of diversity to include in our outreach. These conversations helped us unpack our own social locations and reveal gaps in our established networks. We pivoted to engage policy makers and more people with experiences of structural vulnerability in our education and outreach activities. We also wrestled with how to provide an accessible and inclusive online environment and address power imbalances, considering how to compensate participants, how to ensure we shared a common language and how to balance priorities of different stakeholders.
Unlike traditional evaluation approaches that require specific and measurable goals to be achieved by a step-by-step process, our Hub follows a set of core principles that emphasize authentic engagement of diverse stakeholders, taking time to listen for understanding and moving forward when participants and communities are ready for system change. 1  Using DE in codesign can be a long-term, ambiguous process.
Specifying the scope of our evaluation was challenging with the relative uncertainty in our process and outcomes. We had many questions to ask and the volume of data we could collect had to be limited by our finite resources of time and money. Quantitative data such as numbers of attendees were relatively easy to document, while qualitative data about systems change was more challenging and begged us to ask, 'Who decides what the important metrics should be?'.
We grappled with the need to provide syntheses of data in a timely and digestible way for project/program decision makers and funders while respecting the time required for the meaningful inclusion of community stakeholders. We struggled with power differentials when deciding how to prioritize issues raised by stakeholders who had different opinions on what was important and how to proceed. We encountered tension between academic priorities for knowledge production and education as well as community needs for practical change in health and social services. We were challenged to meet funding requirements for research deliverables while advocating for institutional change and community inclusion in ethics review processes, grant applications and compensation guidelines. A DE approach allowed us to identify issues of power and address these tensions in a respectful, timely fashion so they did not defeat our collective efforts towards inclusive health and social care reform. One of the outcomes of the DE process was a graphic representation of our vision for change, with people at the centre and intersecting priorities that we will build on and track over time.

CONCLUSIONS AND FUTURE POTENTIAL
Our evaluation experience showed us that DE is a useful tool to nurture critical reflection for purposeful research and proactive dialogue in complex system change initiatives. We used the findings from our DE to inform our research priorities in collaboration with EDITORIAL | 1187 community stakeholders. Like codesign in health and social services, a DE approach follows a process that is emergent, flexible and iterative.
We realized that we did not know enough to set specific goals or measurable targets for the Hub in advance, and that we needed to be responsive to the needs and priorities of our stakeholders to ensure that the Hub reflected the principles of authentic and meaningful community engagement in codesign.
Our use of DE provided an opportunity for our team and community stakeholders to build a collective vision and strategies for implementation. Other researchers engaging in codesign work can use DE to engage diverse stakeholders in developing processes and outcomes. Innovative codesign operates in a complex and rapidly changing world; DE tracks the effects of these efforts as they unfold and fosters adaptation according to what is learned.
Our DE approach has helped us see where we were, where we are now, and where we might want or need to go. DE offered a unique evaluation approach that linked complexity and system theories, with empirical inquiry and direct engagement with diverse stakeholders to ultimately promote system change. It is not, however, an approach without limitations. It required commitment to evaluation, openness to critical reflection and resources.
Like codesign, a DE approach requires investment in relationship building at the start of a project/program to negotiate expectations and roles and build trust. It takes time for honest and deep reflection, which can help address issues of power and complexity. It is also an emerging discipline that works outside the normal boundaries of funder/grantee relations that often require clearly planned activities and outcomes.
With an increasing interest in codesign innovations, DE holds promise as a mechanism to bridge the flexibility requirements of codesign work with the accountability needs of sponsors. This editorial is a call for consideration of DE as an evaluation approach to advance codesign. We need more examples and research on the application of this approach to help us better understand its potential and utility for engagement, education, and innovation for inclusive system change.