{'fraction_non_alphanumeric': 0.03375775144020842, 'fraction_numerical': 0.011007962426154918, 'mean_word_length': 5.012353849547761, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Remote mental health services: a mixed-methods survey and interview study on the use, value, benefits and challenges of a national video consulting service in NHS Wales, UK

Social distancing laws during the first year of the pandemic, and its unprecedented changes to the National Health Service (NHS) forced a large majority of services, especially mental health teams to deliver patient care remotely. For many, this approach was adopted out of necessity, rather than choice, thus presenting a true ‘testing ground’ for remote healthcare and a robust evaluation on a national and representative level. Objective To extract and analyse mental health specific data from a national dataset for 1 year (March 2020–March 2021). Design A mixed-methods study using surveys and interviews. Setting In NHS mental health services in Wales, UK. Participants With NHS patients and clinicians across child and adolescent, adult and older adult mental health services. Outcome measures Mixed methods data captured measures on use, value, benefits and challenges of video consulting (VC). Results A total of 3561 participants provided mental health specific data. These data and its findings demonstrate that remote mental health service delivery, via the method of VC is highly satisfactory, well-accepted and clinically suitable for many patients, and provides a range of benefits to NHS patients and clinicians. Interestingly, clinicians working from ‘home’ rated VC more positively compared with those at their ‘clinical base’. Conclusions Post 1-year adoption, remote mental health services in Wales UK have demonstrated that VC is possible from both a technical and behavioural standpoint. Moving forward, we suggest clinical leaders and government support to sustain this approach ‘by default’ as an option for NHS appointments.


BACKGROUND
The pandemic resulted in a paradigm shift in the provision of mental health services due to mandatory social distancing laws. 1 2 From March 2020, the UK, along with the National Health Service (NHS) observed a significant decrease in access to face-to-face appointments, and as a result, an increase in remote services. 1 2 One common remote method for conducting appointments with patients was the use of video consulting (VC). 3 VC can be comparable to face-to-face contact, 4 and in some instances, a preferred method to face to face. 5 VC within mental health services has been internationally utilised for decades, yet the unprecedented circumstances of the pandemic brought to light its widespread ability, use, value, benefits and challenges to countries such as the UK. 1 2 The use of VC permits services to continue across a wide range of mental healthcare conditions, appointment types, sociodemographic groups

STRENGTHS AND LIMITATIONS OF THIS STUDY
⇒ This is a mixed-methods study exploring the remote delivery of mental health services across all Health Boards within one UK nation-Wales. ⇒ COVID-19 presented an opportunity within National Health Service (NHS) Wales to implement and evaluate video consulting in large numbers, across a wide range of NHS services. ⇒ This study provides both a patient and clinician comparison and a comparison across child and adolescent, adult and older adult mental health services which adds value to the current literature. ⇒ Due to COVID-19 restrictions, the study of video consulting has been limited to survey and interview methodologies. However, as restrictions lift, measures around efficacy and effectiveness are now underway, this paper acts as a basis for the new evaluation and research currently being conducted in this area. ⇒ Limitations of this study include small sample size in older adult patients; limited demographic participant information; and that the survey data are likely to be from some recurring patients, thus potentially skewing the sample.

Open access
and health condition status. [6][7][8][9][10] Furthermore, it is sometimes considered more suitable for reaching underserved and isolated populations. 2 VC is reported to provide quality ensured, yet cost-effective mental health support, care and treatment, 11 while reducing patient waiting times and the likelihood of Did Not Attends (DNAs), number of admissions and lengths of stay in hospitals-ultimately relieving pressure on NHS staff and services. 12 However, there is often some concerns expressed regarding the use of VC services within mental health, as it is felt that rapport may not be effectively built, or there is a larger risk of missing out on certain cues or clinical signs, compared with face-to-face appointments. 1 4 7 AIMS The aim of this paper is to present 1 year of remote mental health service delivery data as an evidence base from a UK Wales national and representative sample, using mixed methodology datasets. This paper will discuss findings on the use, value, benefits and challenges of remote services via the method of VC, which is based on the perspective and experience of patients and clinicians across a range of child and adolescent mental health services (CAMHS), adult and older adult mental health services. The findings will be discussed by comparing CAMHS and adult mental health service in dataset 1, comparing patient and clinician feedback in dataset 2. This is combined in dataset 3.

Design, setting, participants
This paper is a mixed-methods study, which combines findings from three datasets from the NHS Wales VC Service. 13

Sampling and recruitment
The sampling approach used for surveys and interviews was opportunity sampling, due to access of the VC intervention and ability to attach a survey link, and access to VC user emails to contact for an interview. There is risk of some bias of those willingly completing the surveys or taking part in interviews as potentially more 'positive' or more 'negative' towards the VC intervention, potentially missing out more 'neutral' responses. Therefore, additional efforts were taken to reduce this bias, and snowballing sampling was also explored via adverts on social media platforms (@teccymru) and through other personal or professional networks for additional interviews. The surveys appeared as internet browser pop-ups at the end of each VC appointment-one per clinician and patient, and were completed immediately as live data to reflect the use, value, benefits and challenges of VC. A copy of the surveys can be found as online supplemental appendix 1). The interviewees were recruited in the course of filling in the post-VC surveys, participants were invited to fill in their contact details if they were willing to be interviewed and this was subsequently carried out by a research assistant, or were contacted directly by the TEC Cymru team via an email invite.

Sample size
A total of 3561 participants were extracted from the NHS Wales VC Service national dataset as representing mental health data. This mental health data subsample is discussed in this paper. Based on the total number of mental health VC appointments conducted during this time (n=7350), the response rate is almost 50%. The data include sections from the following three datasets.

Survey 1 data
Measures on 'use and value'; date period: March-August 2020; total mental health respondents (N=1027).

Follow-up interviews
Themes on 'use, value, benefits and challenges'; date period: June to December 2020; total mental health respondents (N=81).

Recruitment and approvals
Full consent was obtained from all participants. At the end of each survey, a statement of consent and compulsory tick box was required. At each interview, full consent was verbally obtained. TEC Cymru obtained full service evaluation approval and risk assessments for all evaluation conducted in associated to the use of the NHS Wales VC Service.

Measures
In survey 1, respondents were asked questions to capture measures on 'use and value' of VC. There were six questions: quality rating of VC; prevention of the need for face-to-face contact; patient demographics; patients' usage; clinicians work/home location and type of appointment. For analysis of this dataset, the data were split between CAMHS (n=87 patient responses and n=161 clinician responses) and adult services, (n=298 patient responses and n=481 clinician responses) in Open access order to compare and contrast. In survey 2, respondents were asked questions to capture measures on 'benefits and challenges' when using VC. This data are presented as a comparison between patients and clinicians. For this dataset, there were 1563 patients (CAMHS 156; Adult 1383) and 890 clinicians. Unlike survey 1, these data were discussed in comparison of patient and clinician perspectives only, rather than comparing and contrasting across services, due to differences in sample sizes across CAMHS and adult services. Please note that each response in surveys was counted rather than the number of people; a person actively using VC for multiple appointments and filling in the survey after each encounter could be counted more than once. The final section of the paper presents a narrative overview of the experience of VC in mental health services which is broken down into thematic categories, which includes data from interviews and free text narrative captured in surveys.

Statistical methods
Survey 1 data were analysed in terms of distributions of responses, as well as differences that exist between specific groups of respondents (patients and clinicians). Statistical tests of differences were conducted to compare groups on the quality ratings of VC. The quality rating was measured ordinally, as set responses were given for respondents to select. Mann-Whitney U tests were carried out to test the statistical differences between two groups of respondents. Survey 2 data were analysed using descriptive methods of percentages and matched with qualitative data that was manually extracted and thematically analysed to explore emerging themes.

Patient and public involvement
No patient involved.

Participants and setting
A total of 3561 participants were extracted from three NHS Wales VC Service national datasets as representing mental health data. Across all three datasets, the participants included both NHS Wales patients across age and gender groups and mental health conditions, and clinicians across a wide range of mental health service settings in Wales (see online supplemental appendix 1 for full list).

Dataset 1
Use and value of remote mental health services In survey 1, respondents were asked questions to capture measures on 'use and value' when using VC. These data are split between CAMHS (n=87 patient responses and n=161 clinician responses) and adult services, (n=298 patient responses and n=481 clinician responses) to compare and contrast. Table 1 displays the patient age ranges.

Patient demographics
Only 17% of the overall total number of responses were from patients in CAMHS compared with 83% in adult and older age mental health services. There was also a larger percentage of females (60%) compared with males (38%; 2% preferred not to say). Due to small numbers from older adult services alone, the remaining sections combine all adult together, as there were no differences identified between groups.
Quality rating Using a Likert scale ranging from 1-star ('poor') to 5-stars ('excellent') respondents rated the quality of the VC, with a total of 82.3% rating it as 'excellent', 'very good' or 'good'. Considering patients and clinicians separately, patients were more positive in the quality ratings, when compared with clinicians (figure 1), and this difference was found to be significant by a Mann-Whitney U (U=102889.0, p<0.001).
When comparing responses for CAMHS and Adult services, these were very similar in their quality ratings, and no significant differences were revealed by statistical testing (U=93 324.0, p=0.649). 86.6% CAMHS (n=246) and 81.0% of adult (n=773) respondents rated VC 'excellent', 'very good' or 'good'. The responses are displayed in figure 2.

Face-to-face prevention
Respondents were asked whether the VC prevented the need for a face-to-face appointment, with the response options 'yes' or 'no'. 87.8% respondents stated that faceto-face was prevented, which was a positive outcome, given the risks of attending a clinical setting during the COVID-19 pandemic. When comparing CAMHS and adult services, face-to-face prevention was also similar, 81.4% in CAMHS and 89.8% in adult services. Table 2 displays the responses to whether patients had used VC before, how many times and whether they would use it again or after COVID-19 had passed. This reveals that over half of the patients in both CAMHS and Adult services had used VC prior to their  figure 3.

Appointment type
The most common appointment type that was being conducted using VC was 'therapy' (47.5%), followed by

Dataset 2
Benefits and challenges of VC In survey 2, respondents were asked questions to capture measures on 'benefits and challenges' when using VC. These data are presented as a comparison between patients and clinicians. For this dataset, there were 1563 patients (CAMHS 156; Adult 1,383) and 890 clinicians. Figures 4 and 5 display the variation of benefits of VC and figures 6 and 7 display the challenges of VC rated by the patients and clinicians. The highest rated benefits (based on scores between 'very beneficial' and 'beneficial') from the patient are 'lowered risk of infection' (89%); 'no travel or parking', 'better for the environment' and being 'more convenient' (all 83%). The lowest rated benefits were 'better family support' (42%); and 'no time off work and school' (56%)-but these two also rated the highest as non-applicable benefits.
The highest rated benefit (based on scores between 'very beneficial' and 'beneficial') from the clinicians is 'lowered risk of infection' (95%); 'no travel or parking' (86%) and 'more efficient use of clinical time/space' (84%). Interestingly, the lowest rated benefits were benefits they perceived for the patient, including 'improved family support' (42%); reduced likelihood of a DNA (64%) and 'reduced waiting times' (67%).
The highest rated challenge for the patients (based on scores 'a lot' or 'some') was for the 'preference of a face-toface appointment' (24%). Interestingly, patients who stated a strong preference for face to face also rated all other benefits lower, suggesting that VC is more likely to be preferred  Open access by patients who stack other benefits higher, and therefore perceive the value of VC higher. The lowest rated challenge for patients were 'lack of privacy' (2%) and 'unsuitability' (3%). The highest rated challenge for the clinicians (based on scores 'a lot' or 'some') was for poor 'audio/sound' (26%) and 'video/picture' (16%). The lowest rated challenge was 'clinicians lack of confidence' (1%) and 'patient clinical suitability' (4%).

Travel saved and home working -patient and clinician
Each patient was asked, by minutes and miles, how long would they have typically travelled to their appointment if attended in-person. Based on the survey 2 dataset (completed by 1469 patients during 1 September 2020 and 1 March 2021), many patients reported to have saved on travel, with a total of 1254 hours of travel saved, which on average suggests that a patient saves approximately 51 min per mental health NHS appointment (excluding trying to find parking). In addition, each clinician was asked if they are working from home, to calculate the average workforce location per day and how much travel they have saved. Based on the dataset (by 472 clinicians during 1 September 2020 and 1 March 2021), 53% of clinicians reported to be working from home, and to have saved on travel, with a total of 608 hours of travel saved, which on average saves 77 min per clinician, per day in travel time (excluding trying to find parking).

Dataset 3
Narrative overview This final section of the paper presents a narrative overview of the experience of VC in mental health services. For easier reading, the quotes and thematic categories are referenced as: CAMHS patient (CP); adult patient (AP); older AP (OAP); CC; AC; older AC.
The mental health experience: perspective of patients The use and 'preference' of VC in mental health services varies across patient groups and mental health conditions (see online supplemental appendix 1). For some patients, they 'still prefer face-to-face' [AP], feeling that 'VC   An interesting relationship between preference of VC use and the perception of value is commonly influenced by the measure of VC benefits and its challenges. For example, it is frequently reported by patients on the 'ease' of use, stating that VC is 'much easier to use than I had anticipated' (OAP), and 'I feel like it's better on video call as there are so many benefits and it's super easy to use' [AP]. The benefit of 'convenience' for mental health patients in particular, was rated highly, generally due to 'not having to leave the house' [CP]  In addition, patients report how VC enables 'independent care', by 'managing my own symptoms' [OAP], being provided with virtual 'strategies for self-care' and providing 'really good insight'. One older adult patient states that 'It was easy to talk and describe my feelings and emotions using the video call and I was able to understand more about my condition' (OAP). Similarly, a parent of a CAMHS patient says that it 'gives significant insight to patient symptoms and behaviours… a more agreeable and clearer understanding' [CP].

doesn't work well' for certain needs [AP] and that 'face-to-face is better, [because] video feels cut off' [CP]. An adult patient describes how '…the video supported me… but it is still quite a sterile environment and face-to
From the perspective of the mental health clinicians, compared with telephone calls, VC provides additional clinical flexibility, such as the use of 'visual aids', and increases the range of available observational data for assessment of mental state, physical health and mood. It also provides the opportunity to communicate visually through body language, eye contact and facial expressions, and is 'able to make a good connection with the patient, able to pick up non-verbal body language and facial expressions. 'Video quality and sound were both good' [AC] and 'clinically, I am really pleased with the way the emotions of my client was able to be shared' [CC].
Despite the many benefits, it is not uncommon to see patients report technological restraints such as 'unstable internet'; 'audio stalling'; 'glitchy video' or 'frozen screen' and at times, device incompatibilities such as the inability to 'turn the camera front facing' [CP, AP]. For clinicians, the restraints tend to be more specific such as, 'I don't like position of webcam on top of desktop pc screen -hard to look at patient on screen and camera at same time' [CC] or that it is 'difficult to be as clinically effective due to the lag in communication' and 'not being able to pick up on the interpersonal nuances' [AC]. And for others, they find ways around the difficulties, such as 'due to technical issues with sound we had to move to a phone call on but we kept the visual going' [CC].
It is likely that we will need to continue to gather data to analyse whether communicating via VC is as good as face-toface for all mental health treatments and population groups and to see what, if any, therapeutic impact this has on assessment and treatment, particularly where additional reasonable adjustments for communication are required or to accommodate the needs of groups such as younger children who communicate through play rather than speech. Now that the technical infrastructure is improving and many patients and clinicians have both increased their skills and confidence in the use of VC, the potential for offering the choice of routine and sustained use of VC in mainstream mental health provision looks very promising, where clinically appropriate.

DISCUSSION
VC has been in existence for many years, and has an emerging evidence base for its use in healthcare, including mental health. However, it was not embraced as a form of mainstream NHS healthcare delivery in the UK until the COVID-19 precipitated sudden and large-scale change. This change has been out of necessity rather than choice. Traditionally preferring face-to-face methods, often with scepticism among clinicians that VC cannot replace face-to-face contact, particularly for encounters which require high levels of engagement such as therapy, or visual observation of body language and micro-expressions, such as first assessments. 15 This mixed-methods study using both surveys and interviews is, to our knowledge, the most in-depth national evaluation of the use, value, benefits and challenges of VC for mental health remote appointments in the UK, from both patient and clinician perspectives. Our data suggest that VC has a highlevel of satisfaction and acceptability to both patients and clinicians, but also that patients embrace VC Open access more than clinicians. It also suggests that for many patients, VC is a preferred option to face-to-face contact. However, it is important to acknowledge that this is not the general consensus, and that due to the pandemic we may be experiencing a temporary NHS appreciation bias, where there is gratitude for receiving a service at all or that VC preference is being driven out of a temporary risk of infection. Our findings suggest that overall, despite voluntary participation in the feedback surveys and interviews which may have introduced some bias and missed representation, VC clearly demonstrates in the evidence that we have available to us, as well accepted within mental health services and, by all age groups among CAMHS, adult and OAP. Further research and evaluation is needed to explore this in the long-term, including which mental health interventions and conditions are best suited to VC, perhaps looking specifically at VC, blended, or face to face only approaches and how mental health provision, commissioning, training and practice will evolve as a result. Considering how VC also impacts on socio-cultural factors, health inequalities, digital poverty and the need for reasonable adjustments are also important research areas moving forward. It will also be important to continue to measure key metrics such as equity of access, miles travelled, patient outcomes and experience measures for both patients and clinicians across these approaches to compare their quality over time.
The advances due to VC constitute a shift in how NHS mental healthcare services function; this is a transition away from a long-established traditional face-to-face care paradigm which has often placed the onus on patients to attend appointments at the healthcare provider's premises. This shift is also in line with governmental aspirations to increase the use of digital technology and accessibility of healthcare services. [16][17][18] As the opportunity to return to face-toface work emerges, it will be important to encourage a culture of open, transparent and joint decision making between the patient and clinician about the clinically appropriate choice of VC, blended or face to face, rather than returning to services where faceto-face appointments are the expected and default norm. Mental health services have a unique opportunity postpandemic to offer increased patient choice, reflected by need and preference, where flexibility of mode of delivery is seen as a core part of care planning.

LIMITATIONS
There are many strengths to this study, including its mixed-methodology approach, and its relatively large and representative sample for Wales. However, the study did have some limitations. For example, the small sample of OAP impacted on the group being analysed separately in survey 1 and the small sample of CAMHS data in survey 2. There were also limits on available participant demographic data, which provides limited context within the results. Furthermore, due to the voluntary and anonymised nature of the survey data collection, it is unclear as to how often recurring participants completed the survey, thus potentially skewing the sample.

CONCLUSION
This article presents real data gathered from operation of a national VC service, concerning use, value, benefits and challenges encountered by the patients and clinicians in mental health services in Wales. There are cost savings associated with utilising VC, and it could be particularly useful for those patients who find it difficult to leave their homes (eg, severe anxiety, caring responsibilities), have long distances to travel to see clinicians face to face, or simply feel more comfortable with VC. As we emerge from the COVID-19 pandemic and services begin to go back to normal, it would be important to retain VC as an alternative to face-to-face appointments, allowing for patients and clinicians to be able to choose their modality according to personal preference and clinical suitability.
Twitter Gemma Johns @teccymru and Oliver John @olliejon Contributors GJ contributed to the main design of the study and development of the research questions, the main structure and write-up of the paper, and final amendments to the manuscript. JW, BW and GJ analysed the data, with AA, SK and MO supervision. All authors discussed and interpreted the data once analysed and helped structure the manuscript. AB, JT, OJ and AA contributed to the clinical understanding of the findings and shaped the discussion, conclusions and recommendations. AA was responsible for overseeing the full development of the study design and data collection, the analysis and development and final sign-off of manuscript from a clinical and programme perspective. All authors contributed to proof-reading and amendments of the final manuscript.
Funding Technology Enabled Care (TEC) Cymru and its NHS Wales Video Consulting Service is funded by the Welsh Government (no award number provided).

Competing interests None declared.
Patient consent for publication Consent obtained directly from patient(s). Provenance and peer review Not commissioned; externally peer reviewed.

Ethics approval
Data availability statement Data are available on reasonable request. All data relevant to the study are included in the article or uploaded as online supplemental information. All analysed data is published on the TEC Cymru website in the format of a full report of all data for the public to view. To access this reports please see https://digitalhealth.wales/tec-cymru. Other data can be requested as a reasonable request to the corresponding author.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines,