{'fraction_non_alphanumeric': 0.029375380034805946, 'fraction_numerical': 0.016165894366049524, 'mean_word_length': 4.9151680515936995, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 9, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Accessibility in 360o Videos: Methodological Aspects and Main Results of Evaluation Activities in the ImAc Project

This article presents a global overview of the results of the evaluation activities performed in the European project Immersive Accessibility (ImAc), which investigated how access services can be integrated into 360o videos. More specifically, the paper presents the methodological approach in ImAc testing, and reports on the results of the evaluation activities on the tools (a subtitling editor, an audio description editor, a sign language editor, and an accessibility content manager), on the media player, and on the access services (subtitling, audio description, audio subtitling, and sign language interpreting).


Introduction
360º videos are an example of virtual reality (VR) content that allows audiences to immerse themselves in different stories and be exposed to new experiences. According to a report on the use of VR by public broadcasters in Europe (EBU, 2017), almost half of the respondents offer 360º video content. However, it remains to be seen how this content can be made accessible for all. In other words, further research is needed on how access services offered in traditional 2D media, such as audio description (Maszerowska, Matamala and Orero, 2014), subtitling (Matamala and Orero, 2010), or sign language interpreting (McDonald, 2012), may be adapted to the new medium to suit end user needs. This was one of the research questions that the Immersive Accessibility (ImAc) project aimed to answer.
ImAc was a 30-month H2020 European project led by i2Cat which started in 2017. The project was developed by an interdisciplinary team of engineers, audiovisual translation and accessibility experts, broadcasters, and end user representatives. Partners included universities (Universitat Autònoma de Barcelona-UAB, University of Salford), broadcasters (CCMA, RBB) and broadcaster's research centres (IRT), user associations (RNIB), and SMEs (Anglatècnic, Motion Spell). The project aimed to research how access services could be integrated into 360º videos both from the perspective of professionals producing accessible content and from the perspective of end users consuming accessible content. To that end, a series of prototypes were developed and tested in order to come up with the most suitable solutions for 360º accessible video creation and consumption. The specific aim of this article is to present the rationale behind the evaluation plan and the methodological choices made in the ImAc project and summarise the main results. Although some of the results have been partially reported in other publications-and this is duly acknowledged where relevant-, the innovation in this article lies in the presentation of a global overview of how accessibility could be approached in 360º videos.
Section 2 reports on the methodological approach in ImAc and summarises the evaluation plan. Section 3 reports on the evaluation activities with professional tools. Section 4 presents the evaluation activities on the media player interface which gives access to the accessible content. Section 5 discusses the evaluation activities on the different access services presentation modes. The article ends with a conclusion and suggestions for future research.

ImAc's Methodological Approach: Defining Users, Measures and Methods
When designing ImAc's methodology, the first step was to define what evaluation activities needed to be performed, with whom and how. However, this question could only be answered in the framework of the aims our evaluation wanted to achieve.

Aims of the evaluation activities
The ImAc project aimed at developing tools to create access services and producing accessible 360º videos following a human-centric methodology, i.e. by involving the users throughout the design and evaluation process. Therefore, evaluation activities to inform the technological development needed to be planned. In terms of production, the aim was to identify the needs of professionals when creating access services for 360º videos and assess different prototype tools developed taking into account their requirements. In terms of reception, the aim was to identify the needs of end users when consuming access services for 360º videos, to produce requirements, and to assess different solutions. Beyond the specific solutions tested in the different evaluation activities, there was an underlying broader aim: to define how access services could be integrated in the new medium of 360º videos in a moment in which these new formats are being developed (Catapult Company, 2018;EBU, 2017). In other words, the aim was not to research and integrate accessibility as an afterthought, but to find the means to produce 360º videos which are "born accessible", to use Orero's (2020) terminology, and which comply with the EU current legal framework for accessibility.

Defining users and metrics
Since we are concerned with the production and consumption of access services, two broad categories of users were identified at the beginning of the project: the so-called "professional users" and "home users". Professional users, which is the term used in the project to refer to producers or creators of access services, would be involved in evaluation activities concerning the professional tools. Home users, which is how the end users of the access services were referred to in ImAc, would be involved in evaluation activities to assess the access services features. The challenge lay in the fact that accessible 360º video content is non-existent and, therefore, most home users are not familiar with this environment. This is why, in the first stages of the project, users with technological expertise were prioritised in a series of focus groups (Agulló and Matamala, 2019;Agulló, Matamala and Orero, 2018) that laid the foundations for the evaluation activities that are described in this article.
Concerning the methodological approach, it was decided that a combination of standard metrics and open questions would allow us to triangulate results from the evaluation activities. The evaluation approach was based on three core concepts: usability, presence, and preferences.
When using a tool to professionally produce accessible content, or when using a tool to access content, usability is fundamental. We understand usability as the "extent to which a product can be used by specific users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use" (ISO 9241-11:1998, definition 3.1.). In order to measure the usability of the different ImAc tools, the System Usability Scale (SUS) was considered a fast and reliable metric. SUS includes 10 items, and it is considered easy to administer and reliable with small sample sizes (Brookes, 2013).
When developing solutions to integrate access services in virtual environments, it was considered critical to develop solutions that would not impact negatively on the user's presence. Presence, also referred to as immersion, can be understood as the sense of "being there" (Reeves, 1991), the "perceptual illusion of non-mediation" (Lombard and Ditton, 1997), the "psychological sense of immersion in any mediated environments" (Fryer and Freeman, 2012), or "the experience of being engaged by the representation of a virtual world" (Jacobson, 2002). Presence is a multiconstruct concept with many dimensions that can be measured by various means. In our case, the Igroup Presence Questionnaire (IPQ) (Schubert, Friedmann and Renbrecht, 2001) was chosen for three main reasons: it combines previous questionnaires and differentiates for the first time between spatial presence, involvement, and experienced realism; it has been validated in virtual environments, and its length (14 questions) is adequate for experimental purposes.
In both cases, when developing solutions for creation and for reception, subjective feedback and general preferences of participants were also considered to be fundamental. Hence, open fields and general preference questions were included in the different evaluation activities.
The results of the different evaluation activities allowed ImAc to identify the best approach to produce and consume access services in 360º. These solutions were then implemented in two open final pilots, led by German broadcaster RBB and Catalan broadcaster CCMA, in which the evaluation measures chosen were mostly related to quality of service and quality of experience metrics. These final pilots will not be reported in this article, as our interest lies in the evaluation process prior to them.

Evaluation protocol
Evaluation activities in ImAc were carried out by different partners. To guarantee a unified approach, an evaluation protocol to be followed in any evaluation activity was developed, with the following steps. First, participants are welcomed, they are informed about the project and its ethical procedures, as approved by the UAB's ethical committee. Ethical consent forms are signed. Secondly, participants are asked to fill in a short demographic questionnaire. This questionnaire, with slight variations for professional and end users, was consistent across all evaluation activities. Thirdly, participants are asked to perform certain tasks and reply to the selected questionnaires. Finally, participants are thanked and are offered the possibility to obtain more information about the project.

Evaluation plan
ImAc evaluation activities can be categorised into three blocks: a) activities to evaluate the tools that professionals will use to create the services, b) activities to evaluate the media player, which home users will use to reproduce the content, and c) activities to evaluate different access services features. The selected access services were subtitling (ST)-including a hybrid service: easy-to-read subtitles-, audio description (AD), audio subtitling (AST), and sign language (SL). Table 1 provides an overview of all evaluation activities.

Evaluating professional tools
ImAc developed four tools: a subtitling editor (Figure 1), an audio description editor ( Figure  2), a sign language editor (Figure 3), and an accessibility content manager ( Figure 4). An accessibility content manager is a type of software that can be used by a broadcaster to keep track of the creation of access services for certain content. For example, the content manager can be used to assign video content to professionals and keep track of what access services have already been produced for each video asset.
Initial development for ImAc's tools was informed by a series of focus groups (Agulló and Matamala, 2019;Matamala, 2018, 2021) which gathered initial user requirements. A first prototype was then developed and tested for the tools (except for the SL editor). This evaluation round allowed the consortium to collect more user requirements and suggestions and develop a second prototype, which was again tested (see picture on the right on figures 1, 2 and 4).

Methodology
Evaluation activities on the tools focused on two aspects: usability, by means of SUS, and user preferences, through an open questionnaire which read: • What did you like most about the (tool)?
• What did you like less about the (tool)?
• What do you think could be improved, and how? • Did you miss any functionalities? If so, can you tell us which ones? • (for editors) Do you find the features of setting the angle easy to use? Explain why.
• (for editors) Were the preview modes useful for you? Explain why.
• (for editors) Do you think it will take longer to subtitle/audio describe/interpret videos in 360º? Why? • (for editors) Do you think 360º videos will impact your work as a subtitler? • (for ACM) Was it intuitive? If not, please write why. • Other comments (open field).
The protocol was as follows: first, participants were informed about ImAc and the specific test, and informed consent forms were signed. Then, participants were asked to fill in an online demographic questionnaire and to perform a series of tasks included on an instructions sheet. SENDEBAR (2021), 32, 65-89. https://doi.org/10.30827/sendebar.v32.16881 After finishing these tasks, they were asked to fill in the SUS questionnaire and a preference questionnaire, both online, and were thanked for their participation.
The number of participants was 131 professionals, who provided objective and subjective data on the different ImAc tools. Sociodemographic information about these participants was obtained through a standardised questionnaire, including: age, sex, main language, current job, previous experience with 360º content and with access services, level of studies and specific training, software use, technological use (devices used on a daily basis, virtual reality consumption), and attitudes towards virtual reality. A detailed description of this demographic information is beyond the scope of this article but can be found in Matamala (2020). It should be stressed that involvement of participants was more challenging for some tools, such as the SL editor, because current professional sign language interpreters do not generally use this type of software, hence the difference in numbers. Table 2, for both the first and second iterations, where applicable. In all cases an improvement is found from the first to the second iteration, showing the impact of user feedback on tool development. However, results are not always above average-68 or more is considered above average in SUS-and the values for the SL editor are low. Possible reasons that explain these results are the fact that many audiovisual translation professionals, especially audio describers and sign language interpreters, do not always use tools to produce their access services, contrary to content and accessibility managers using the ACM. For instance, many describers taking part in the test stated that they are normally in charge of writing the audio description using text processing software, but they are not actually involved in the recording, which was one of the tasks performed in the test. This was even more striking in the SL editor test, where participants stated that they do not usually deal with these processes and acknowledged the challenge of familiarising themselves with a new tool. This highlights the need for further training in technological tools for future professionals in the field, together with an even more thorough analysis of the specific needs of professionals.

SUS results are presented in
Beyond quantitative results, what was more interesting for ImAc was to gather qualitative data from users on the specific tools, but also on how professionals envision the impact of 360º videos in their processes. The results presented next do not deal with specific functionalities (see Agulló, 2020, and, for details) but provide a broad overview of what professionals consider important in professional tools.
The analysis shows that professionals are willing to work online, with cloud-based tools, with a clear user-friendly interface and visible icons. Visual design is critical but at the same time very much influenced by participants' previous experience with other tools. Professionals want to be able to customise the user interface but also the different shortcuts to make their work more efficient, and they are interested in editors that integrate additional tools such as spellcheckers or quality assurance tools. They want the tool to provide automatic features (for instance, subtitle segmentation) but, at the same time, they want to be empowered to adapt these automatic tools to their needs and ignore them if necessary. Time-coding is fundamental, as well as the presence of a clear sound wave and multiple options to edit the services and to export the final result to different formats. Overall, ImAc prototypes were defined very often as "intuitive" and "easy to use", but room for improvement was also identified and implemented as the project progressed.
Some of the most praised functions were related to navigating in a 360º environment and setting the angle. In fact, setting the angle is a specific feature that needed to be developed for 360º videos. In 360º content, speakers may not be located within the audience's field of view, but somewhere else in the sphere. This is why the position of the speaker needs to be indicated for each subtitle or sign language interpreted section. To do so, the professional needs to navigate in the video and select the angle where the speaker is positioned in order to anchor the access service to the speaker's position. This metadata will then make it possible to implement different features, such as positioning the subtitle/sign language interpreter close to the speaker or activating a guiding mechanism such as an arrow or a radar that will show the viewer where to find the speaker. Similarly, in audio description settings the angle allows professionals to link the audio description to the place in the sphere where the action being described is taking place, generating metadata that will make it possible to produce audio coming from that specific region.
Beyond the specific features of the editors, what is interesting from a global perspective are the attitudes of the professionals towards a new technological development such as 360º videos. When asked if they thought that producing subtitles, audio descriptions, or sign language interpretations would take longer in 360º, the replies were: in the first iteration, 61% thought that subtitling would take longer due to the need to set the angle, 26% of the subtitlers were not sure, and 11% stated that it would not take longer with the right tools. In the second iteration, the percentages were 50% for those who considered it would take longer, 16% for those who thought it would not take longer and 34% for those who stated that it would take more time only at the beginning, when getting used to this new process. Concerning AD, 79% of the professionals in the first iteration and 70% of the participants in the second iteration saw AD 360º content as more time-consuming due to the level of visual details available and the need to set the angle. As for sign language interpreters, 50% thought it would take longer, 25% considered it would depend on the content, and 25% thought that, with the right tools, it should not SENDEBAR (2021), 32, 65-89. https://doi.org/10.30827/sendebar.v32.16881 have an impact. Overall, it seems that professionals acknowledge the challenges of producing accessible 360º content but also see the need for the right tools to make this process easier.
When asked about the expected impact of 360º videos on their professional work, their attitudes were the following: 42% of subtitlers in the first iteration and 50% of the subtitlers in the second iteration think 360º videos will have an impact, although maybe not immediately. 19% in the first iteration and 17% in the second iteration do not think 360º videos will impact their professional work, and the remaining participants do not have a clear idea about it and state that it is highly dependent on how 360º videos evolve in the market. Regarding audio describers, 58% in the first iteration and 39% in the second iteration think it can have an impact, even if maybe not in the short term, but 21% and 35% do not agree with this view. The rest are not sure and express concerns.
With regards to sign language interpreting, 33.32% think it will have an impact, 49.98% indicate that it will not, at least yet, and 16.6% express uncertainty. Overall, there is some uncertainty about the impact of this new format on the professional work of access service professionals and a feeling that this may happen sometime in the future.

Evaluating the player
ImAc developed a player (Hughes and Montagud, 2020) to enjoy the 360º accessible content and tested different versions. On the one hand, two tests with users with hearing loss were performed in Spain and Germany. The aim was to compare the performance of a standard menu-what in the project was called "traditional menu"-when used on an HMD and on a tablet. The first iteration took place in September-October 2018, and the second iteration in May-June 2019 ( Figure 5). On the other hand, a test with users with sight loss, both in Spain and the United Kingdom, evaluated an enhanced accessibility menu ( Figure 6) and a voice interaction system ( Figure  7) with Amazon Echo. The enhanced accessibility menu was developed taking into account the needs of participants with partial sight loss, who may require bigger icons. This test was preceded by a pilot test in which the methodology was tested. SENDEBAR (2021), 32, 65-89. https://doi.org/10.30827/sendebar.v32.16881

Methods
Testing the player interface focused on two aspects: usability of the tools, by means of SUS, and user preferences, which were gathered by means of a set of questions adapted to each evaluation action and by some additional open questions: • What did you like most about the ImAc player/menu interaction?
• What did you like less about the ImAc player/menu interaction?
• What do you think could be improved, and how? • Other comments.
The protocol was the same as in previous tests: first, participants are informed about ImAc and the specific test, and informed consent forms are signed. Then, participants are asked to fill in an online demographic questionnaire and to perform a series of tasks: play and pause the video, change the volume, activate access services, personalise them, etc. After finishing these tasks, they are asked to fill in the SUS questionnaire and a preference questionnaire, and are thanked for their participation.

Participants
The number of participants in the traditional menu test in Germany was 12 participants in the first iteration, in 2018, and 12 participants in the second iteration, in 2019. In Spain, the number was 13 in the first iteration, in 2018, and 12 in the second iteration, in 2019. In respect of the enhanced accessibility menu versus voice interaction test, data from 23 users was obtained in Spain in 2020 (19 voice interaction/9 enhanced accessibility) and data from 28 participants SENDEBAR (2021), 32, 65-89. https://doi.org/10.30827/sendebar.v32.16881 was gathered in the UK in 2019-2020 (15 voice interaction/9 enhanced accessibility). Pilot tests were performed in all cases, but they are not reported here. Detailed demographic information about the participants is available in Matamala (2020).

Results
Table 3 provides SUS scores for the different tests concerning the player. The player interface evolved during the project, but most versions assessed obtained above average SUS values. A strict comparison of the results across countries and iterations is not possible, as the participant profile was not homogeneous (for instance, mean age in the German second iteration was higher than in the first one) and the user interface was localised into the different languages.
In the tests with users with hearing loss, the extensive personalisation and the clear design were considered useful features. Users provided positive feedback on the fact that they could choose a guiding mechanism, be it an arrow or a radar, to locate the speakers in the 360º sphere when the speakers were not visible.
Most users were able to perform the requested tasks. However, it was observed that some of the personalisation features were named on the menu with terms that were not clear enough and needed to be changed. Similarly, the universal icons that were used in the interface were not clearly identified by participants, who suggested adding an abbreviation to clarify their meaning. Further suggestions for improvement and specific bugs were also identified during the tests. For instance, when users activated the subtitles in tablet mode, the changes were not immediately visible but hidden behind the main menu, which caused some confusion. This and other aspects were improved in the final version of the player.
Moving beyond specific details linked to the specific user interface versions tested, it was observed that many participants found the HMD uncomfortable if they were wearing glasses, cochlear implants, or hearing aids. The users wearing the latter two devices asked whether it was possible to stream the audio directly to their devices. Overall, most users reported not having had any experience with 360º content before, because accessible 360º content is almost not available, but they were amazed by the experience and saw the potential for documentaries and concerts, for instance.
Regarding users with sight loss, they tested either the enhanced accessibility menu or the voice interaction, depending on their needs. Concerning the enhanced accessibility menu, they SENDEBAR (2021), 32, 65-89. https://doi.org/10.30827/sendebar.v32.16881 provided specific advice which was very much related to their sight condition and to the size, colour, and bordering of different menu items, putting forward the need for user customisation. Regarding the voice interaction, most participants found that interaction with the ImAc player through Amazon Echo was easy to use, but also expressed frustration with the specific phrasing that needs to be used. They thought more conversational language should be used, an aspect to be further developed by voice assistants. The differences in assessments across countries were maybe due to other factors such as the development of the artificial voice used and previous experiences with Amazon Echo. It should also be highlighted that blind users complained about using an HMD, which they found cumbersome.

Evaluating the access services
Different presentation modes were tested for each of the access services. They are reported independently in the following subsections.

Subtitling
The main focus when testing subtitles in 360º was on two central topics: where to position the subtitles (subtitle positioning), and how to guide the viewers to the speaker in the 360º sphere (guiding mechanisms).
When users do not have access to the audio, and the speaker is not in their field of view, mechanisms to guide them to the speaker's position are needed. Otherwise, the user may lose part of the action. In ImAc, an arrow and a radar were developed and tested in different iterations ( Figure 8). Subtitling positioning is also critical to avoid losing dialogue. In ImAc, different options were tested: first of all, the so-called always visible with arrow ( Figure 9) was tested against fixed positioned with arrow ( Figure 10). Fixed positioned subtitles are attached to the speaker, which means that if the speaker is not on the field of view, the subtitles will not be shown. This is why a guiding mechanism is added, in this case an arrow ( Figure 10). On the contrary, always visible subtitles in ImAc are subtitles displayed in front of the viewer, following the viewer's movements. They are equivalent to static-follow subtitles by Brown et al. (2017Brown et al. ( , 2018 and to static subtitles by Rothe et al. (2018). Secondly, the always visible ImAc subtitles were tested against the subtitling positioning mechanisms used in other platforms such as the BBC (n.d.) or The New York Times, namely subtitles equally spaced in a fixed position, in this case every 120º (Figure 11) (Agulló andMatamala, 2019, 2020). Additionally, the innovative concept of easy-to-read subtitles was put to the test with ImAc content. Easy-to-read subtitles (Bernabé and Orero, 2019) are simplified subtitles which follow Easy-to-Read Language standards (Bredel and Maaß, 2016;IFLA, 2010;Inclusion Europe, 2009) and are validated by users . Other minor tests related to the use of emojis or textual information and related to the comfort field of view were also performed but will not be reported here in detail.

Methods
Subtitling testing focused on presence and preferences. The aim was to make sure the ImAc solutions did not have a negative impact on presence whilst considering user preferences. The protocol was in line with the other ImAc protocols: welcoming participants and providing information, signing informed consent forms, and watching two randomised clips where each of the presentation modes were shown. After watching each clip, participants were administered the IPQ questionnaire and at the end were asked to reply to a general questionnaire with slight variations. In all cases there was a question about what system they preferred and a question in which they had to explain why. They were also asked what could be improved, and how. Some additional questions were included. For the guiding mechanism test, these questions were: • Would you implement another system to guide you to the speaker? • How easy was it to identify who was speaking in the clip with the arrow system? (Likert scale) • How easy was it to identify who was speaking in the clip with the radar system? (Likert scale) • Do you think you will be able to enjoy 360º videos with this type of subtitles? Explain your answer.
For positioning, the additional questions were: • How easy was it to read the subtitles in (option a/b)? (Likert scale) • How easy was it to find the subtitles in (option a/b)? (Likert scale) • How easy was it find the speaker in (option a/b)? (Likert scale) • Did you like the blinking effect on the arrow? Yes/No.
For easy-to-read subtitles, the additional questions were: • How easy was it to read the subtitles in (option a/b)? (Likert scale) In all cases, participants were also invited to provide additional comments. Pilot tests were performed to test the methodology but are not reported here.

Participants
The number of participants was as follows: • Test on guiding mechanisms: 12 participants (Germany) and 13 participants (Spain) in 2018.
All were persons with hearing loss. The material used was an excerpt of the short film I, Philip. • Tests on positioning: • Always visible with arrow versus fixed positioned with arrow: 12 participants (Germany) and 12 participants (Spain) in 2019. All participants were persons with hearing loss. The material used was an excerpt of the short film I, Philip. • Always visible versus equally spaced in a fixed position: test in Spain, in 2018, with 40 participants: 20 persons with hearing loss and 20 hearing persons, as it was considered that subtitles were useful for all user profiles regardless of their hearing loss, in line with Agulló, Matamala and Orero (2018). The material used was the documentary series Holy Land.
• Test on easy-to-read subtitles versus standard subtitles: 36 participants, in Spain, in 2019. The targeted participants were elderly and their mean age was 69.4 . The material used was Romeo and Juliette, a 360º opera in French subtitled into Spanish.

Results
Results were gathered on presence and preferences for both guiding mechanisms. The ImAc solutions did not present a statistically significant difference in terms of presence, implying that both options have a place in 360º and their usage can actually be a matter of preference. Table 4 summarises all median values on a 7-point Likert scale and provides a summary of the statistical tests performed. A Wilcoxon Signed-Rank test was chosen for evaluation activities where we wanted to assess whether the distribution of two paired variables in two related samples was the same. This is a common non-parametric test for data that may not follow a normal distribution. A planned comparison test (T-test) was chosen to compare preferences between easy-to-read subtitles versus standard subtitles by elderly participants (Spain 2019b), because of the number of participants. An effect was only observed when comparing the always visible ImAc solution and the fixed positioned every 120º solution which is found on other websites. Table 5 presents mean values and a summary of the statistical analysis. In terms of guiding mechanisms, arrows performed better than the radar system, and were also the preferred system in all tests ( Table 6). The reasons for preferring the arrows were related to the fact that the arrow was easier to identify and understand and less distracting. The radar was considered more visually obstructing and less intuitive. However, for some users, the radar was better when there were many speakers at the same time, as it gave more accurate spatial information. SENDEBAR (2021), 32, 65-89. https://doi.org/10.30827/sendebar.v32.16881 Concerning subtitle positioning, the always visible solution was better accepted than the ImAc fixed positioned solution or the fixed positioned found in other broadcasters such as the BBC. The reasons expressed by participants relate to the fact that in this mode they do not miss the content and subtitles are easier to find. Fixed positioned subtitles force the user to move their head to follow the information and are considered more challenging. However, there are some challenges related to a possible dizziness effect that some participants choosing the fixed positioned every 120º reported for the always visible mode. These participants also stated that fixed positioned subtitles were easier to read (see Matamala, 2020, for details on the 2018 Spanish tests). Finally, in the easy-to-read (E2R) subtitles versus standard subtitles for the deaf and hard of hearing (SDH) ( Table 8), data indicate that preferences are distributed equally, which shows the potential of this new mode, especially with the elderly. When asked whether the subtitles were easy to read and easy to understand, easy-to-read subtitles always obtained better values. It remains to be seen what the results would be in other context beyond opera, where the language may be literary and the focus is put on the musical performance.

Audio description
Audio description provides an auditory explanation of what is seen on screen, in other words, it is an intersemiotic translation of images into words. There were two AD tests: in a first round, a pilot test was performed with a reduced sample of participants (5 in the UK and 6 in Spain) to test three AD modes based on sound treatment: Classic, Static, and Dynamic. In the Classic mode, the ambisonic sound is placed above the user's head. In the Static mode, the sound is located on the user's side, as if someone were sitting close to the user, telling the story. In the Dynamic mode, the sound is placed at different angles, depending on where the main action being audio described takes place. The rationale behind this mode was that it would allow users to locate the events within the 360º sphere more easily, hence feeling more immersed in the story. The results of this test were not as expected and showed us that partici- pants felt confused with this approach . They were more interested in the script features, and some of them actually required more extended audio descriptions. Taking into account the feedback from the pilot test, we revisited the AD modes and suggested three new approaches: Classic, Radio and Extended. In all three modes the sound treatment stays the same-it is placed above the user's head-but the scripting style changes. Classic AD offers a standard approach to AD. Radio AD features an unconventional approach, with more engaging and informal language. Extended AD allows users to pause the video and listen to an additional description, activated at their will.

Methods
Testing focused on presence and preferences. The protocol followed was the same as in other ImAc tests: welcoming participants and providing information, signing informed consent forms, filling in a demographic questionnaire, and watching clips. In this case, they listened to one general audio introduction to the content and then watched three episodes of the series Holy Land with randomised audio presentation modes. After each video they replied to the IPQ questionnaire and, at the end, they were given a general questionnaire with preference questions. This questionnaire included the following questions: • Which AD type do you prefer in 360º? Rank the AD modes according to your preferences.
• Please explain why you ranked the AD modes in this way.
• How could AD be improved in this medium? • Other comments.
At the end of this test, participants stayed for one additional test on audio subtitling, which is reported later in this article.

Participants
All participants were persons with sight loss recruited through user associations. 31 participants took part in the UK in 2019-2020 and 30 participants took part in Spain at the beginning of 2020.

Results
Results on presence are summarised in Table 9 and results on preferences are presented in Table 10. Paired samples statistics T-tests did not present any statistically significant difference (all p>.05), showing it is a matter of preferences. In fact, regarding preferences, Table 10 summarises the main results. The two innovative approaches offered by ImAc were the preferred ones by participants, although with differences: whereas participants in the UK preferred the radio version, participants in Spain preferred the extended option. UK participants choosing this option reported being frequent users of radio dramas and audiobooks, which may adopt a similar approach. It is remarkable that 76% of the participants in the UK and 66% of the participants in Spain chose the classic mode either as their first or second option. This does not come as a surprise, as this was the format that was more similar to what is currently offered on standard television in both countries.

Audio subtitling
Audio subtitles (AST) (Braun and Orero, 2010) are written subtitles which are read aloud either by a human professional or a text-to-speech system. This access service is especially useful for those who cannot see the subtitles (for instance, persons with sight loss) and for those who cannot read them (for instance, persons with reading difficulties). It can be offered independently or it can be integrated in the audio description. To gather data on this access service, participants in the AD tests (see above) were asked to perform one task in which they had to state their preference between AST only or AST with AD in a very specific type of content: opera.

Methods
Testing audio subtitling was part of the same experimental session in which the AD was tested. To avoid participants' fatigue, the test focused only on preferences, with the following questions: • Which type of AST do you prefer in 360º? AST/AST-AD. • Explain in your own words why you prefer the option chosen in the first question.
• What system would you prefer? AST with no subtitles on screen./AST with subtitles on screen that can be enlarged./I would like to select when I want subtitles on screen and when not. • How could AST be improved in this medium? SENDEBAR (2021), 32, 65-89. https://doi.org/10.30827/sendebar.v32.16881 • In what excerpt did you feel more present? Only AST./AST-AD./I did not feel present in any of them./ I felt equally present in both. • Other comments.
The content used for the test were two clips from the opera Romeo and Juliette, with a length of approximately 4 minutes each. An audio introduction was offered before the test to contextualise the clip.

Participants
The same participants as in the AD test took part in this test (see section above). Table 11 provides a summary of the results both in the UK and in Spain. Results indicate that preferences and self-reported presence differ in the two countries under study. It should be highlighted that the stimuli were translated and voiced by different talents, which can have an impact on the results. It is also possible that cultural factors and different habits in terms of accessible opera consumption may be behind the choices of the participants. Opera audio subtitling and audio description is a reality in Spain, but users in the UK are not very familiar with this practice, hence this may explain why Spanish participants were more willing to accept AST than British participants.

Sign language interpreting
The aim of the SL interpreting tests was to address three main aspects: a) display of signer video, which could be continuous-meaning the signer window would be visible all throughout the content-or non-continuous-meaning the signer window would only be visible when interpreting was needed; b) presentation of sign language only versus presentation of sign SENDEBAR (2021), 32, 65-89. https://doi.org/10.30827/sendebar.v32.16881 language plus subtitles; c) speaker representation, which could be performed through an emoji representing the speaker or a textual description.

Methods
The protocol followed was the same as in other ImAc evaluation activities. In this case, three tasks were performed, addressing each of the items above. For each of the tasks, one clip per condition was prepared and their presentation was randomised across participants. The stimuli for the tasks on signer video display and for the tasks on speaker representation were extracted from I, Philip, whereas the videos for the second task came from the opera Romeo and Juliette. The test was developed at RBB in Germany, and content was translated into German and German Sign Language. For the first task, presence was measured through IPQ and preferences were assessed using the following questionnaire: • What system do you prefer?
• Please indicate why you prefer the above-indicated option.
• Please indicate why you did not choose the other option.
When participants were administered the SL with subtitles versus SL only task, only preferences and self-reported values were gathered, by means of three questions: • How easy was it to read the information with SL and subtitles/SL only? (7-point Likert scale) • How easy was it to understand the information with SL and subtitles/SL only? (7-point Likert scale) In the last test, and to avoid participants' fatigue, a single question with the same scale was put to participants: How easy was it to know who was speaking with the textual representation/emoji representation?

Participants
10 Deaf participants took part in the test. They were recruited through German broadcaster RBB. Their main languages were German (2) and German Sign Language (8).

Results
As far as the presentation of continuous or non-continuous sign language interpreting is concerned, median scores for each subscale on the IPQ questionnaire are shown in Table 12. In terms of preferences, data from one user could not be retrieved due to technical problems. 6 out of 9 users prefer a non-continuous display because it is easier to concentrate on the video content. Those participants who preferred the continuous display felt reassured by this option and thought it reproduced a real-life situation, where the interpreter is always present.
The results of the comparison of SL with subtitles versus SL were unexpected by the researchers, as Deaf communities often express a strong preference for SL and are reluctant to use subtitles. When participants were recruited, they were asked whether they would like to test subtitles or SL, and the participants on this specific test chose SL. Still, when confronted with signed content that also displays subtitles, four users indicated that they preferred to use SL only and six users indicated that they preferred to use both services. Some of the advantages offered by the combination of access services, according to the test participants, are: the colour of the subtitles helps them to identify the speaker and, especially in the opera content, the subtitles transport more information than the SL interpreting. When asked how easy it is to "read" the video with SL and subtitles and the video with SL only, 80% of the responses were on the 5 to 7 range on a 7-point Likert scale, where 7 equals "very easy". When asked how easy it is to understand it, the video with SL and subtitles received values between 5 and 7 in 80% of the responses, whereas the video with SL only had 60% of the replies in the same range. Finally, when asked to indicate their level of agreement with the statement "I think reading SL and subtitling simultaneously is overwhelming for me", only 30% selected the maximum value.
Regarding the last task, 7 users preferred the textual speaker representation and only 3 chose the emojis. They reported that the text enables an easier identification of the speaker. This is undoubtedly a topic that still needs further research, as this was only assessed with one question at the end of the test.

Conclusions
ImAc has taken one of the first steps in researching accessibility in virtual environments. Evaluation activities have been performed when VR is still not widely spread and when both professional and home users are not used to this technology. This implies many challenges but, at the same time, many opportunities. The different evaluation activities presented in this paper have allowed us to map the initial needs and requirements of professionals producing accessible content and gather interesting input on their views. There is still much uncertainty about the impact 360º videos will have on their professional lives, but many of them envisage this technology positively, albeit acknowledging that producing access services in VR will be more time-consuming.
In respect of home users, tests with end users have allowed us to evaluate different prototype versions of the player. The player has received positive usability assessments and an innovative voice interaction feature has been welcomed by participants with sight loss, although they would like to interact with more conversational language. Regarding access services, subtitling tests have put the emphasis on positioning and guiding mechanisms, showing that ImAc solutions perform better than existing solutions, and also showing a user preference for the arrow as an indicator. However, it remains to be seen if this behaviour is kept when users SENDEBAR (2021), 32, 65-89. https://doi.org/10.30827/sendebar.v32.16881 are confronted with narratologically complex scenes in which different speakers are found at different places in the 360º sphere.
Additionally, the tests have put forward the potential of applying easy-to-read principles to subtitling with the elderly, although there is a need to expand this testing to other user groups and content typologies.
Regarding audio description and audio subtitling, two access modes generally addressed to persons with sight loss, script-based evaluation has shown the potential of new approaches beyond the classic mode. Again, replication with other types of content and future tests with improved sound features are still much needed.
Finally, sign language testing has identified user preferences in the presentation of the interpreter video and has put forward an unexpected result, i.e. that a significant percentage of users prefer to have both SL and subtitling, even if they are native SL speakers.
Overall, tests have not shown statistically significant results. Although the sample size may have had an impact and further tests with wider samples are needed, it seems the ImAc solutions have a similar effect on presence, which makes us think users should be given different choices according to their needs and preferences. Customisation is undoubtedly the future of access services in media consumption and more research is needed in the field. Schubert, T., Friedmann, F., & Regenbrecht, H. (2001). The experience of presence: Factor analytic insights. Presence: Teleoperators and Virtual Environments, 10(3), 266-281.