{'fraction_non_alphanumeric': 0.0291385904206884, 'fraction_numerical': 0.008407697444302798, 'mean_word_length': 4.942821067821068, 'pattern_counts': {'":': 2, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
"Look, all our hard work is paying off": A qualitative evaluation of a system-wide, workforce development model to promote positive behavioural support.

BACKGROUND
Positive behavioural support (PBS) has been identified as a means of improving the quality of life and support of people with an intellectual disability. This qualitative study explored the views of service providers about a regional PBS programme, that was underpinned by a workforce development approach.


METHOD
Semi-structured interviews were conducted with senior staff (n = 42), from organizations which provided services to people with an intellectual disability, about their views about, and perceived impact of, the PBS programme, Data were analysed using thematic analysis.


RESULTS
Two themes were identified: "It's what PBS is about, isn't it?" emphasized the importance of the programme being systemic- and evidence-based; "Macro to micro" highlighted the different points in the systems of care at which the programme was seen to impact.


CONCLUSIONS
We discuss the results in the context of the study limitations.

understanding of CB as learned behaviours and the role of applied behavioural analysis in informing assessment of, and interventions for, CB. The third addresses process issues, such as the need for interventions to be function based, include reactive and proactive strategies, and be underpinned by functional analysis which is informed by robust data.
Positive behavioural support approaches have generally been shown to be effective (e.g. Bowring, Totsika, Hastings, & Toogood, 2020;Lewis et al., 2019;MacDonald & McGill, 2013;McGill et al., 2018;Positive Behavioural Support Coalition UK, 2015), although not all studies have found it to be so (e.g. Hassiotis et al., 2018) and providing staff training in PBS can result in positive outcomes, such as increased staff knowledge and confidence, reductions in CB (see MacDonald, McGill, & Murphy, 2018;O'Dwyer, McVilly, & Webber, 2017). There is, however, a limited focus on wider outcomes such as quality of life or staff practice (but see Dench, 2005;Hassiotis et al., 2018;MacDonald et al., 2018;McGill et al., 2018).
Likewise, despite PBS being a multi-component, systemic approach, and mediator analysis being a key component of PBS approaches (PBS Coalition, 2015), most approaches have used a staff training, rather than a systemic workforce development (WFD) approach in order to promote change. A WFD approach differs from a more traditional staff training approach by taking account of the systems within which an individual works, rather than just focusing on the individual learner. A WFD approach is likely to consider and address organizational and structural factors, as well as individual training needs and is often underpinned by collaborative, strategic partnerships between different stakeholders (Jacobs & Hawley, 2009).
The idea of a need for a systemic approach to PBS is not a new one. School-wide PBS approaches have existed in the US for a number of years (see Horner et al., 2007), and in 2013, Allen et al., emphasized the need for a whole-system PBS approach in the UK, noting "The task here is to design the broader culture so that a PBS approach is systematically supported at all levels." (p38). Since that time, other authors have highlighted the influence on CB and PBS of the wider organizational and social systems within which staff and those being supported operate (Olivier-Pijpers, Cramm, & Nieboer, 2019). A few researchers have taken account of some of these wider factors in their PBS interventions (e.g. McGill et al., 2018). Others have highlighted that systemic factors such as managerial inconsistency in accommodating for the increased workload of those volunteering to train (Hassiotis et al., 2018), lack of time to discuss behaviour support plans in team meetings (McKenzie, MacLean, Megson, & Reid, 2005)  In 2018, a comprehensive PBS programme was developed in the North East of England. This was underpinned by a WFD model (McNall, 2012), and shaped by the results of an earlier WFD needs assessment of the existing PBS practice and requirements in the area (McNall, McKenzie, & Branch, 2016), as well as research into what people with an intellectual disability and their families (McKenzie et al., 2017 considered to be good quality support, with a focus on PBS. The programme, a collaboration between NHS organizations, a local university and other stakeholders involved the development of three accredited PBS programmes, offered free of charge to staff in social care organizations who provided support to people with an intellectual disability and/or autism; and three WFD manager roles. The remit of the WFD managers included providing support and supervision to, and assessing the knowledge and competence of, those undertaking the programmes, as well as facilitating system-wide culture change by liaising with stakeholders, developing communities of practice in the local areas and contributing to PBS developments, such as revising commissioning specifications.
The overall project was funded by the NHS and overseen by a steering group. An independent evaluation of the programme was also commissioned, and this paper reports on one aspect of this.
The accredited programmes were based on a cascade structure and provided a mixture of face to face teaching, e-learning and supervision to three cohorts of staff, which led to one of the following qualifications: A Postgraduate Certificate or Advanced Diploma in Leading PBS (Level 6/7 award); Certificate in Facilitating PBS in teams or Award of competence in PBS for support workers (Level 4 awards). Table 1 provides an overview of the programme content and structure. Each module lasted for 3 months, and each topic had associated e-learning materials which were written specifically for the programme and which took the students between 40 min and 2 hr to complete. Every module also had three full day teaching sessions to supplement the e-learning materials, and every student received direct supervision that followed a competency-based assessment related to the PBS academy standards.
The level 6/7 students received their supervision from the WFD managers, all of whom were PBS specialists, while the level four facilitators received supervision from the person within their organization who had completed the level 6/7 course. These facilitators then offered supervision to the level 4 support workers, again using a competency-based assessment tool. A community of practice was established in each of the three main localities in the region to further support the development of the staff and a shared understanding of core principles. The Level 6/7 cohort comprised of service managers and leads, the first Level 4 cohort comprised of senior support staff (facilitators), and the second Level 4 comprised of front line support staff. The Level 6/7 staff provided support and supervision to the facilitators, who in turn provided this to the front line staff, with the WFD managers also providing ongoing supervision and support.
The PBS programme targeted system-wide as well as individual change, with the ultimate aim of improving the service provision and quality of life for those being supported. This paper reports on a qualitative evaluation of the impact of the model. As a WFD approach was used, this evaluation explores the views of stakeholders in wider NHS, commissioning, social care and other services, as well as the L6/7 students who had undertaken the PBS qualification.

| Design and ethics
Ethical approval was obtained from the first author's university ethics board. A qualitative methodology was used. Data from semi-structured interviews were analysed using thematic analysis. This method of analysis was chosen because, while it offers a rigorous method of exploring the subjective experiences of individual participants, unlike other methods of qualitative data analysis, such as Grounded Theory or Interpretative Phenomenological Analysis, it does not require a particular theoretical or epistemological stance (Braun & Clarke, 2006) and does not have the development of theory as an aim (Tie, Birks, & Francis, 2019). This allowed the flexibility to account for any differences in the epistemological views of the research team, although the authors who were primarily involved in the analyses adopted a constructivist approach and had as their focus the subjective perspectives of the participants about the PBS programme.
The core research team comprised two clinical psychologists/ researchers who were experienced in working with people with an intellectual disability and/or autism and PBS and who had previously published research in this area, and two experienced research assistants. The latter were psychology graduates, who had been involved in the evaluation of the PBS programme for over a year and were, therefore, knowledgeable about the aims and context of the programme. The core research team members were not involved in the development and delivery of the programme and were commissioned to conduct an independent evaluation. The remaining authors were an experienced clinical psychologist and a specialist in WFD, both of whom were involved in the development and delivery of the PBS programme. The context of the study was the evaluation of the impact of the PBS programme outlined in the introduction.

| Participants
Forty-two individuals participated, of whom 27 had undertaken the Level 6/7 programme and 7 who were undertaking, or due to begin the Level 4 programmes. The remaining participants included two social work professionals and staff who held senior roles in support organizations or held regional or national strategic roles in relation to the service provision for people with an intellectual disability. Information about individual work titles is not provided to prevent participants being identified. Eleven participants were male, and 31 were female. Ages (where reported) ranged between 27 and 55.

| Procedure
Students on the programme, who had already consented to take part in the evaluation of the PBS programme, were asked to participate either during a PBS Workforce Development Impact event or shortly after the event. All others were recruited at the PBS event, which was designed to provide information and feedback to senior staff about the PBS programme. Participants were provided with information about the study at the first presentation of the PBS event and invited to come to a separate area of the venue throughout the day to meet with the researchers if they were interested in participating. Here they provided written consent and were interviewed.
Interviews were audio or video recorded and were conducted by the first, second and third authors (an experienced clinical psychologist and two psychology graduates), transcribed by the third author and analysis was carried out by the first and fourth authors, both of whom are experienced researchers and clinical psychologists.
The interviewers all used the same interview schedule, to ensure that the same broad areas were covered in each interview, while also allowing the flexibility to explore themes that were important to individual participants in more detail. The interview schedule covered the following questions: the role of the participant, their relationship to, and involvement with, the PBS programme, the impact of the programme (both positive and negative) in general and in relation to specific areas. The latter included questions about the personal impact (e.g. on their knowledge or practice), their organization (e.g. on policies and procedures), on the support of people with an intellectual disability (e.g. changes in activity levels, CB, other aspects of quality of life), on others (e.g. families, friends), on wider systems (e.g. on how services were commissioned) and in any other areas (e.g. financial impact).
The development of the guide was informed by a number of factors including research and guidelines which highlighted the areas that might be expected to change if PBS was being used consistently and well, for example improved quality of life, changes in staff practice, reduction in CB (Dench, 2005 The interviews lasted between 10 and 40 min. The shorter interviews were all with those participants at the impact event who had not had direct involvement with the PBS programme and so had a more specific focus compared with others who could provide a perspective on a wider range of impacts of the PBS programme.

| Analysis
Analysis was conducted in accordance with the guidance from Braun and Clarke (2006). All responses were combined into a single data set, with the responses from each participant being initially read, reread and coded individually. These individual codes were then combined and considered in the context of the entire data set in order to generate themes and associated subthemes. The analysis was shared with the wider research team to ensure it was consistent with the perspective of all those who had conducted interviews. Potentially identifying information was removed to ensure confidentiality.

| RE SULTS
Two themes were identified: "It's what PBS is about, isn't it?" highlighted the importance of the development and delivery of the programme being consistent with key PBS concepts; "Macro to micro" highlighted the many levels at which the programme was reported to have impact. These themes, and their associated subthemes, are outlined in detail below.

| Theme one: "It's what PBS is about, isn't it?"
This theme reflects the importance that participants placed on the programme being consistent with PBS principles and values, in particular that it was underpinned by an evidence base and reflected stakeholder views, as outlined in the subtheme: "That evidence base…it proves its worth." The systemic WFD nature of the programme was also seen as important to its success, as this reflected the range of factors that influence behaviour and culture change, which paralleled the systemic, multi-component nature of PBS. This is outlined in the second subtheme: "Thousands of peoples' lives have been touched."

| "That evidence base…it proves its worth"
The fact that the programme was underpinned by research was seen as important as a means of reflecting some of the key principles of PBS-stakeholder involvement and evidence-based practice: The underpinning research base was also seen as a way of ensuring that the programme had quality, credibility, was fit for purpose and jus-

| "Thousands of peoples' lives have been touched"
The importance of the current PBS programme being underpinned by a WFD model, which reflected the systemic nature of PBS, was identified by many participants. It was recognized that individual improvements in staff knowledge and skills, which occurred as a result of the programme, would be ineffective without a wider infra-structure to support them: Thousands of peoples' lives have been touched in some way by this program and it is certainly something that we need to promote and spread. (P38)

| Macro to micro: the impact of the programme
This theme reflected the many levels at which the programme had resulted in positive change. The related subthemes reflect these different levels and include "It was nowhere near": individual change, which reflects the ways in which the participants experienced changes in their knowledge, understanding and confidence; "Embedding PBS: organizational change" which outlined new systems, processes and policies that resulted from the PBS programme; "Look, all our hard work is paying off" which described the impact on staff teams and "It actually works" which highlights the impact on those being supported and their families.

| "It was nowhere near:" individual change
At the individual level, the participants were aware of changes in their knowledge and increased understanding of what PBS was: "I thought I had a fair bit of insight, but it's helped me realize that there actually a lot more stuff I needed to learn." (P33). Participants also expressed an increase in confidence, which helped them to implement change and to challenge other professionals when they felt that proposed or existing approaches were not in the best interests of the people they supported: Just last week when he's had his CPA [Care Programme Approach], we've said like we don't think it's appropriate and it's made me more confident to say that because I've got the evidence to say why and he's been taken off that.
(P4) For many participants, their understanding of what PBS was had changed. What they previously had considered to be PBS, they now saw fell short: I think I thought I knew what PBS was at the beginning, but I don't think I really did.

(P5)
The organisation did sort of believe we were following a PBS programme but… it was nowhere near.
Along with this increased knowledge came an understanding that PBS is not about fixing behaviour, it is a value-based approach which aims to improve the quality of life of those who are being supported: It's really kind of helped me really relate back to why we do it and the quality of life for people.

(P13)
Actually, it's not just the challenges that need to be taken into account, it's how the quality of that person's day affects that person still. (P3)

| Embedding PBS: Organizational change
The participants were open about the limitations that they perceived in their organizations as a result of their new understanding of PBS.
Many identified organizational changes that they had already implemented. As most held senior roles they were able to implement changes in strategy, policy, systems and practice: It made me realise that we had lots of system changes to make throughout the company and obviously I'm in the position to do that as the director of the company, so we've already started a lot of the system changes.

(P40)
Many organizations introduced new, or adapted existing, policies to make PBS a central component: "…and we've just for the first time a specific PBS policy for the organisation" (P33). These changes were often part of whole-system changes, which reflected the shift in the organizational culture that the PBS programme had facilitated: "I'm on renewing a policy now…but the whole organisation structure around it will be changing." (P1). The participants reported a focus on changing approaches to recruitment to ensure that the right staff were employed in the first place: We've changed our application forms as well…we're just doing a lot more of looking at people's values, just where before it may have just been down to their education and their experience.

(P11)
There was also a recognition that the principles and value base of PBS extended beyond those being supported by the service and were equally applicable to everyone, including the staff team: It's not just PBS for customers and service users, it's PBS for the staff, it's developing the staff and committing to them.

(P9)
The clients we work with have functions to their behaviours but also the staff do as well.

(P12)
This understanding changed the way that staff teams were supported by the organization, based on an acknowledgement that supporting people with behaviour that challenges can be stressful for staff: It's highlighted areas that we've needed to focus on such as resilience… as well as making sure we've got the right support mechanisms in place. Yeah, staff sickness is a lot better. We don't get staff going off sick as much as we did before. It's just been that obviously it has been time consuming, it has taken us away from our main jobs, it is additional areas of responsibility for staff.
(P20) For most, however, this was accepted as a consequence of participating in the programme. Some identified related benefits, such as increased staff team involvement and quicker resolution of existing time-consuming issues: There has been no negativity in the service that we've started delivering, the behaviour support plan and the staff that have been involved in it. Our only issues are time, there's so much to do.

(P2)
Obviously, it would be ridiculous to say it doesn't create extra workload but for me… I think we have addressed things that have helped reduce some of the behaviours that were taking up a lot of time. (P15)

| "Look, all our hard work is paying off"-the impact on staff teams
The recognition that PBS was applicable to everybody was seen to result in positive changes for the wider staff teams: Their knowledge, skills and expertise have increased dramatically. It's increased motivation… and engagement is much, much higher now. And their own role in maintaining it: When we did his assessments and looked at the ABC charts, the staff were reinforcing his behaviour. (P4)

| 1519
Published for the British Institute of Learning Disabilities MCKENZIE Et al.
These benefits fed directly into changes in practice and the way that support was provided: Yeah, it's definitely changed some of the practice… We're definitely seeing more input into the PBS plans… and people are coming to us with suggestions, improvements and actually reviewing the plans that are in place better.
(P3) So, instead of waiting until instances happen and then going back, people are being really proactive about how they can prevent things from escalating from the start. (P17) The increased use of systematic recording methods, monitoring and review also meant that staff could see evidence of positive changes that their new approaches had brought: It was great to say after a few months "look all our hard work is paying off"…that's a recordable, hard fact. Like a 50 percent reduction in [challenging] behaviour, which is amazing.

(P8)
Staff attitudes towards some of the people they supported had also changed, allowing them to see beyond the behaviour to the aspects of the person that were previously overlooked: We're seeing a side of his personality that we probably didn't see before, he's quite funny actually, he has quite a good sense of humour which we didn't see before cos he was always so angry. (P4)

| "It actually works"-the impact on those being supported
A number of specific examples were given of the ways in which changes in practice had influenced the way that direct support was provided. There was an increase in positive approaches… We put a skill building programme in place to teach functionally equivalent replacement behaviours…So, we now have got this young man who is actively involved in all of his food prep, he has a communication pictorial booth to choose daily activities inside and outside the home, he has a weekly activity structure that links to observation feedback form. So, his community-based participation has increased by 70 percent.
(P19) This is the guy who would get frustrated around mealtimes because he wanted to feed himself, but someone feeds him. So, it's just teaching skills again and all we did was we got some specialist adaptation kind of knives and spoons and forks and kind of hand over hand and give him that kind of skill teaching, and now he feeds himself.
(P12) And a reduction in restrictive approaches: Since we started using PBS in the workplace, restraint has reduced…and the use of chemical restraint has actually decreased also. (P24) We've been able to reduce some levels of restriction in terms of physical intervention for people in PBS plans that are no longer needed.

(P13)
These changes were both reported as resulting in reductions in behaviours that challenged: …and actually yesterday they reported that the number of incidents with this person has actually halved within two or three months which is amazing. The people that we support lives have changed incredibly. They've got more independence, they're happier, they're more, it's, it's, it actually works.

(P29)
The quality of life is much improved, choice, everyday choices, having things to do every day.

(P28)
She's now eating a healthy, well balanced diet, going out in the community, doing stuff that she, she wasn't doing before so yeah, she's doing really well.
(P32) A significant area where the programme had impacted the quality of life of those being supported was through relationships with families. This happened directly, for example, through organizations engaging in better ways with families: I think we always thought that we involved families and parents but I think we really are now and the conversations that we're having around function and what people want, we're making sure that the families are at the forefront of that and involving those, involving them in any interventions that need to happen and I think that's made a really big impact. (P5) We've got the opinions of family of people we support a lot more than we probably ever have before now and the course has been a huge part of that.

| D ISCUSS I ON
The study aimed to evaluate the views of senior staff about, and their perceptions of the impact of, a regional PBS programme. Two aspects were seen as crucial to the success and credibility of the programme, as reflected in the first two subthemes. First, that it was underpinned by a systemic WFD framework that recognized the range of stakeholders and factors that influence behaviour and culture change (McNall, 2012). Second, that the development of the programme was evidence-based, and, in particular, that it reflected the aspects that people with an intellectual disability and/or autism and their families identified as being components of good quality care (McKenzie et al., 2017. In addition, it was based on an assessment of the WFD needs of organizations who were and/or would be delivering PBS in the region (McNall et al., 2016). These factors were seen as reflecting the key components of PBS and, as such, were seen as increasing the commitment to, and engagement with, the programme. The second theme outlined the different levels at which impact had occurred. Consistent with much previous research (see MacDonald et al., 2018), the participants reported increases in confidence and knowledge. There was also a move from seeing PBS as a technique for use with people with an intellectual disability to change CB, to a universally applicable value-based approach that ultimately aimed to improve the quality of life and opportunities for the person being supported . The increased knowledge led to many organizational changes, including to staff recruitment retention and performance management. Common factors which have been found to be associated with staff stress, burnout and intention to leave are a lack of staff support (e.g. Hatton et al., 2001), poor communication and teamwork (Denny, Wells, & Cunningham, 2011) and supporting people who display CB (e.g. Robertson et al., 2005). By contrast, good quality support from well-trained supervisors who provide clear procedures and feedback to staff has been found to be related to better staff retention (e.g. Kozak, Kersten, Schillmöller, & Nienhaus, 2013;Vassos et al., 2013). Wider organizational and systems factors, such as inconsistency in workload management (Hassiotis et al., 2018) and the practice leadership of managers (MacDonald et al., 2018), may also account for the different results found between studies. In respect of workload, high work demands combined with low control over your job is associated with staff stress (Lee et al., 2009). While many student participants noted that participation in the accredited programmes had increased their workload, very few identified this as negative or stressful. This may be because the first cohort all held senior organizational roles and were likely to have had the authority to adjust their own workload and that of other staff to meet these increased demands. In addition, all participating organizations made a commitment at the start of the programme to free up staff time.
A further difference between the present study and previous research was that the WFD managers provided ongoing support, supervision and modelling to staff on the programmes, with senior staff in turn modelling these approaches to the Level 4 staff. Such practice leadership and work-based coaching have been identified as important components of both PBS and Active Support (e.g. Bigby, Bould, Iacono, & Beadle-Brown, 2019;Bosco et al., 2019). The latter, which identifies the strengths and needs of the person and uses targeted support to enable the person to engage in meaningful activities and relationships with others, is argued by some researchers to be both complementary to, and a crucial component of, PBS in terms of supporting proactive approaches (e.g. Ockenden, Ashman, & Beadle-Brown, 2014). As participant 19 notes, the staff support became more focused on observing and shaping practice through rehearsal, modelling and provision of work-based feedback. This model also combines the two components, structured "classroom"-based input and work-based coaching, that have been found to be most effective (van Oorsouw, Embregts, Bosman, & Jahoda, 2009).
The study had some limitations. First, the study reflects the perspective of staff and professionals rather than of people with an intellectual disability. It is the latter group who are best placed to judge if PBS has resulted in an increase in their quality of life. Further research is planned to obtain these views. Second, participants were a self-selecting group and it may be that their views differ from those who chose not to take part. Third, there was some variation in the ways in which participants provided feedback and by whom they were interviewed, which may have influenced the results to some extent. Related to this, the relatively large number of participants, their different levels of involvement with, and roles in relation to the PBS programme (e.g. commissioners compared with front line staff) meant that some themes and subthemes were more pertinent for some than for others. The interview guide allowed flexibility in exploring the most important issue for the different participants, however, and the main themes were consistent across interviews.
A further consideration is that, as the focus of the study was the subjective perspectives of the participants about their experiences of the PBS programme, we did not seek to verify whether their perceptions were based on objective evidence. It may be that the participant perceptions were influenced by factors other than the effectiveness of the PBS programme itself, such as having been involved in any form of change process or having committed some time and effort to the process.
An additional issue to consider is that the analysis of the data was conducted by two clinical psychologists who were knowledgeable about PBS and had conducted previous research in the area. It is likely that their interpretation of the data was shaped by these factors. While the subjective and interpretative nature of qualitative research is acknowledged, in order to try and maximize the rigour of the study, the quality criteria suggested by Yardley (2008) were adopted. First, sensitivity to the theoretical, social and organizational context in which the research was being conducted was addressed by basing the interview schedule and interview process on areas identified by research as being important and relevant. This also confirmed the importance of the topic area and the potential for impact to be identified. Second, commitment and rigour were addressed by trying to ensure that the participant sample was appropriate, that is those with some experience of the PBS programme, who were able to provide their subjective views about it. In addition, the final interpretation of the data, which itself was based on discussion and agreement between two researchers, was shared with the wider research team to ensure that it was consistent with the content of the transcripts and evidenced by the sample quotes. This also helped ensure transparency and coherence of the data.
In conclusion, the results indicate that an evidence-based PBS programme that is underpinned by a systemic WFD model was perceived by a number of senior stakeholders as having resulted in changes that had a positive effect on staff knowledge, confidence and practice; resulting in wider organizational benefits, such as improved strategy and policies, and team stability; and improving the quality of life of those being supported. An important area for future research is to explore what the specific mechanisms were that helped promote these changes.

ACK N OWLED G M ENTS
The authors would like to thank all of the participants and colleagues on the PBS steering group who contributed to the development and implementation of the PBS programme, in particular Judith Thompson, Melissa Sherring, Lynne Jones, Alison Branch and Jill Chaplin.

CO N FLI C T O F I NTE R E S T S
None.