{'fraction_non_alphanumeric': 0.030037054191755442, 'fraction_numerical': 0.01602593793422881, 'mean_word_length': 4.801558511352949, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Comparison of a Head-Mounted Display and a Curved Screen in a Multi-Talker Audiovisual Listening Task

Objectives: Virtual audiovisual technology and its methodology has yet to be established for psychoacoustic research. This study examined the effects of different audiovisual conditions on head-yaw and eye-gaze direction as well as on preference when listening to multi-talker conversations. The study's goal is to explore and assess audiovisual technologies in the context of hearing research. Design: The participants listened to audiovisual conversations between four talkers. Two displays were tested and compared: a curved screen (CS) and a head-mounted display (HMD). Using three visual conditions (audio-only, virtual characters and video recordings), three groups of participants were tested: seventeen young normal-hearing, eleven older normal-hearing, and ten older hearing-impaired listeners. Results: When there were no visual cues, participants tended to look ahead; when visual information was available, they looked at the target talker. Participants generally turned their head close towards the target talkers when presented as video than virtual characters. In addition, an effect of display showed that in the most extreme case when the talker was at 45 degrees, they turned closer when wearing the HMD than when viewing the curved screen. Open interviews showed that the CS was preferred over the HMD for older participants and that video recordings were the preferred visual condition. Conclusions: Using different audiovisual setups may lead to slightly different movement behavior in terms of head yaw and horizontal gaze. These differences, as well as preference for specific audiovisual technologies, should be taken into account when designing and comparing experiments.


INTRODUCTION
In recent years, audiovisual technologies have become more prevalent in hearing research (Seol et al. 2021, Ahrens et al. 2019, Schutte et al. 2019, Stecker 2019, Devesse et al. 2018, Kohnen et al. 2016, Assenmacher et al. 2005, Rummukainen 2016, Hendrikse et al. 2018b). One of the motivations of using such technologies is to increase the ecological validity of the experiments in the laboratory and the clinic (Keidser et al. 2020), i.e., that the results in the laboratory are representative of situations in the real world (Bentler 2005, Cord et al. 2004. This is particularly important when fitting hearing aids for the first time. New users tend to give up using hearing aids if these don't improve their hearing situation in their daily environments, thus leading to a poorer quality of life in the long term (McCormark & Fortnum 2013;Tareq ue et al. 2019).
Audiovisual technologies such as head-mounted displays (HMDs) and surrounding screens are already established and are available in the market. For hearing and hearing aid research, however, the applicability and acceptance of different audiovisual technologies has hardly been investigated. Seol et al. (2021) tested speech perception with and without HMDs and asked the participants about technology preference and its applicability in the clinic. Almost all participants were willing to complete the test in a clinical setup with the HMD, but the weight of the device and the participant's unfamiliarity with it were concerning issues. Seol et al. (2021) mentioned it is crucial to test and validate the audiovisual technology used in audiological experiments, "as it could be one of many factors that professionals and patients would consider before employing and performing the test in clinics". More data are therefore needed to characterize and establish audiovisual technology options in hearing research .
To assess the validity of audiovisual environments for hearing research, movement behavior was found to be an important metric Grimm et al. 2020), as the complexity and realism of a scene changes behavior. Hadley et al. 2021 found that listeners in a conversation adjusted their head orientation in relation to their interlocutor when noise levels increased, presumably to optimize speech intelligibility. Listening behavior was further affected by the type of background noise used in those conversations. In a simulated environment, the quality and realism of the visual cues also changes movement behavior: Hendrikse et al. (2018b) found that as the character animation became more realistic, head yaw and gaze direction approached those obtained using video recordings. Similar results can be found in Carter et al. (2013), where participants looked more at the face of the talker as the visual realism increased. These findings motivate the use of movement behavior metrics to investigate differences between audiovisual setups, as it was done in this work. Head orientation is particularly relevant for directional hearing devices, as sounds can be processed differently depending on their direction of arrival and the user's head orientation . Where the participants are looking when listening, i.e., gaze behavior, is also an important metric, as future hearing devices could include gaze information in their algorithms (Favre-Felix et al. 2018;Grimm et al. 2018;Hart et al. 2009;Kidd Jr et al. 2013).
Individual characteristics, in particular age and hearing type, could elicit different movement behaviors and technology acceptance. Most research in movement behavior and technology acceptance has been done with young normal hearing (YNH) participants, thus data is lacking for older and hearing impaired participants. YNH participants do not systematically rotate their head to improve their acoustic situation in laboratory setups (Grange et al. 2018;Shen et al. 2017), but participants with hearing impairments and advanced age could employ different listening strategies: Hendrikse et al. (2019) found indications that older participants made greater use of head orientation compared to eye movement than YNH participants in everyday-life simulated environments. Regarding technology acceptance and preference, Philpot et al. (2017) found no difference in preference for young adults between the CS system and the HMD when watching a 360-degree angle documentary. Hendrikse et al. 2018b found that YNH participants preferred video recordings over animated tual characters in a listening task. Older and hearing impaired participants might be more reluctant than YNH participants to use intrusive and immersive audiovisual technologies, although the opposite is possible as well. Surrounding screens and HMDs differ in several characteristics that could affect acceptance and listening behavior in laboratory measurements. A HMD is worn on the head and it occludes all visual references to the real space. The field of view is reduced, as current consumer-grade HMDs cannot cover the whole human field of view (210º horizontal). The HMD requires head straps that could interfere with the positioning and performance of the hearing aids, and therefore could be uncomfortable for hearing aid users. Additionally in Seol et al. (2021), HMDs were reported as heavy and difficult to use if one is not familiar with the device. With surrounding screens, the real space is always visible. The user's head movement and vision are less constrained, as no device is worn on the head. To understand such differences between displays and visual conditions, the rent study provides comparative data for different readily ble audiovisual technology options, in particular for curved screens (CSs) vs. head-mounted displays (HMDs), and for video recordings (VID) vs. virtual characters (VC) vs. audio-only (AO). Technology acceptance is particularly important for the bility of such technologies in clinical setups. If we want to use such technologies in the clinic, we have to make sure that older and hearing impaired participants are willing to use such technologies and that these technologies do not deter the participants from getting their hearing abilities checked. In consideration of that, this study included YNH participants, older normal hearing participants (ONH), and older hearing impaired participants (OHI).
The research questions of this study are "Does display type, visual cues, age, and hearing type affect listening behavior?" and "Can we use such audiovisual technologies with older and hearing impaired participants in laboratory and clinical setups?" In this experiment, participants listened to conversations in different audiovisual conditions and answered questions about the content afterwards; listening behavior, characterized by head yaw and horizontal gaze direction, was measured and analyzed; and subjective ratings and open comments of preference and technology acceptance were collected. The results of this study are meant to provide useful insight to guide future research and implementation in hearing clinics and research laboratories using audiovisual technologies.
This study replicates parts of the experimental setup of our previous study Hendrikse et al. (2018b). The current work extends it by adding the HMD as a display type and by testing older participants, with and without hearing impairment. With two display types combined with three visual conditions, a direct comparison between several audiovisual setups is possible. Additionally, older and older hearing impaired participants were included, in order to compare the applicability of the audiovisual setups in clinical environments.

MATERIALS AND METHODS
The participants were asked to listen to conversations under six different conditions: two display types combined with three visual conditions. The display types were a CS and a HMD, and the visual conditions were audio-only, virtual characters, and video recordings. In Fig. 1, the conditions can be seen as they looked in the experiment. The top row shows the conditions with the HMD and the lower row the ones with the CS. The task of the participants was to answer three questions about the content of the conversation they just heard. After completing all six conditions, they had to do an interview and fill out questionnaires.

Participants
Seventeen young normal-hearing subjects (YNH), eleven older normal-hearing subjects (ONH), and ten older moderately hearing-impaired subjects with hearing aids (OHI) participated in the study. All but one of the YNH subjects were students of the Carl von Ossietzky Universität Oldenburg with a mean age of 24 years (STD 2.43,. YNH subjects were specifically asked about their hearing: none of them reported hearing loss. The mean age of the older participants was 61.5 years (STD 5.3, range 50-69). ONH and OHI participants were recruited through Hörzentrum Oldenburg GmbH, where their audiograms were measured regularly: ONH participants had a mean pure tone average (PTA) between 125 Hz and 8 kHz of 10 dB HL; the mean PTA between 125 Hz and 8 kHz for OHI participants was 49.4 dB HL. The OHI participants had been using their hearing aids for more than six months and had a moderate symmetric hearing loss. They wore their hearing aids during the experiment. Participants were also specifically asked about visual impairments, which none of them reported (e.g., reduced vision not corrected by glasses or contact lenses). The ethics permission was granted by the ethics committee of the CvO Universität Oldenburg (Drs. 1r63/2016). The participants signed an informed consent.
Of the thirty-eight participants, the data from thirty-five was used in the analysis of head yaw and gaze direction: the data from one OHI participant was accidentally deleted; one ONH participant could not finish several trials because of a technical error; for another ONH participant, poor electrode connectivity caused too much noise in the gaze data. Thus, for the analysis of head yaw and gaze, we used seventeen YNH, nine ONH and nine OHI participants.

Setup
The experiment was conducted inside a circular 'tent' within an acoustically semi-treated room (reverberation time (T60) = 0.13s). The inside of the tent and a top view of the room can be seen in Fig. 2. The figure shows where the participant was sitting and how the projection looked for the CS-VID condition. The position of the elements of the tent is also shown in Fig. 2. The tent was covered with a black blanket and it had a radius of 1.98 meters. It consisted of a metal structure that supported a circular array of 16 loudspeakers (Genelec 8020B, Genelec Oy, Olvitie, Finland) and an acoustically transparent curved screen. The loudspeakers were spaced every 22.5-degree angle at a radius of 1.96 meters and a height of 1.60 meters. The curved screen was in front of this array of loudspeakers and was 2 meters tall with a 1.76-meter radius. Images were projected onto the screen from a close-field projector (NEC U321H, Sharp NEC Display Solutions, Munich, Germany) placed on top of the tent. The projector achieved a projection of 120-degree angle (horizontal) and had a refresh rate of 60 Hz with a resolution of 1920x1080 pixels. The HTC Vive system (HTC Corporation, New Taipei City, Taiwan) was used as HMD. The HTC Vive Base Stations and a camera for live-feedback were placed above the curved screen. The HTC Vive display had a refresh rate of 90 Hz, a resolution of 1080x1200 pixels per eye, a 100-degree angle field of view (horizontal) and orientation and translation tracking. The background noise level inside the tent with all the devices working was 31.1 dB A. A chair was placed in the center of the tent, facing towards the front, i.e., the 0-degree angle azimuth of the simulation. The chair was on an elevated platform with dimensions 120 cm by 120 cm. The platform was elevated 30 cm from the floor. When the participants were seated, the ears were at approximately the same level as the loudspeakers (1.60 meters). To the side of the participant, around 120-degree angle azimuth from the front, there was an emergency button at arm's reach: pressing this button stopped the simulation.
Three computers were used in the experiment: an Ubuntu 14.04 for the acoustic rendering, data logging and master control; an Ubuntu 14.04 for the screen projection with NVIDIA Quadro K6000; and a Windows 10 for the HMD rendering with NVIDIA Quadro M5000 and head tracking.
The 3D virtual acoustic environment was rendered with TASCAR (Grimm et al. 2019b) versions 0.175.2-0.177.5. The virtual 3D scene for the curved screen was created and rendered with the Blender Game Engine version 2.79 (Roosendaal 1995).
The image warping for the projection was done with the graphics card and was manually configured and calibrated. The 3D scene for the HMD was rendered with the Unity game engine version 2017.1.0f3. All the sensor data was transmitted for central data logging in TASCAR via the Open Sound Control (OSC) (Wright and Freed 1997) and the LabStreamingLayer protocol (Kothe et al. 2018). The experiment was controlled and executed with Matlab 2016b and with the acoustic engine using the OSC protocol. Temporal alignment between visual and acoustic cues was adjusted manually.
Head orientation was measured with two different devices for the CS and the HMD. For the CS, participants wore a head crown with a Vive Tracker (HTC Corporation, New Taipei City, Taiwan) attached. For the HMD, the device itself, i.e., the HTC Vive, was used for head tracking. The devices were tracked and calibrated using the Vive Base Stations and SteamVR versions 1515459962-151967395. The tracking software used the same coordinate system and reference to the real world for both head trackers. We decided not to use the head crown with the Vive Tracker when using the HMD, because it would be uncomfortable for the participant to wear both at once. Niehorster et al. (2017) analyzed the accuracy of the HTC Vive for position and orientation tracking. They found that the RMS error was below 0.02 cm for position and 0.02-degree angle for orientation. The latency of the HTC Vive was 22 milliseconds and the update rate of the head tracking was 120 Hz. They stated that most errors appear when tracking is lost; for this reason, they recommended using the HTC Vive tracking system in which the risk of losing tracking is small. This was the case in the present study, as participants were seated on a chair in the center of the tracking area, and the chance of losing tracking was small.
The horizontal movement of the eyes was measured with two electrodes placed next to the eyes (electrooculography, EOG). The EOG device used a custom-built EOG amplifier (a highimpedance operational amplifier, a 10-bit analog-digital converter, and a built-in first-order high-pass filter to compensate for the electrode voltage drift) and a Bluetooth transmitter (Hendrikse et al. 2018b). The sampling rate was 50 Hz and the horizontal eye angle could be estimated with an accuracy of ±10-degree angle. Calibration of the EOG device was done once for the CS and once for the HMD. The display showed a cross and the participant was asked to follow it with their eyes. The cross was displayed briefly in different positions to force horizontal gaze jumps of 5, 10, 15, 20, 25 and 30-degree angle while we compensated the head rotations in real-time: if the participant turned their head towards the cross, the cross would move away in order to maintain the head-eye angle. EOG data was considered inconsistent for a given trial when it showed out-of-range gaze angles and/or constant saccades (fast gaze shifts) of more than 30-degree angle. For seven participants, one of the two calibrations produced inconsistent results, thus the best calibration out of the two was chosen.

Stimuli
We used the same audiovisual material, casual acted conversations, as in our previous study (Hendrikse et al. 2018b). There are seven conversations available, one of which we used for the training and the six remaining for the experiment conditions. The material can be found in the database by Hendrikse et al. (2018a). The conversations lasted between 1 min 24 s and 1 min 39 s and the topics were food, holidays/travelling, weather, work, future plans, movies and anecdotes. Of the four talkers, two were females and fluent non-native talkers (German CEFR C1), and the other two were males and native talkers. In the 3D virtual scene, the actors were positioned at 45, 15, -15 and -45-degree angle in a radius of 1.7 meters away from the listener's position. After each conversation, one of the actors asked three multiple-choice questions about the content. In this experiment, we did not record the answers of the questions but this was unknown to the participants in order to keep them engaged. These questions can be found in the aforementioned database (Hendrikse et al. 2018a).
Acoustic Stimuli. The acoustic conditions were the same across all trials. The multi-talker conversations were played together with diffuse background noise. In our laboratory, the loudspeaker layout did not match the position of the target talkers (see Fig. 2). We used TASCAR to generate a virtual acoustic environment and to reproduce sound sources at the prescribed place. This virtual acoustic environment simulated a virtual source for each target talker at the predefined position, as if the clean speech was played back through a loudspeaker in the room at the respective position. The audio reproduction technique used for the target talkers was horizontal 7th-order Ambisonics with max-rE decoding (Daniel et al. 1998), rendered to the 16 loudspeakers on a ring at ear level. The diffuse background noise was a 1st-order Ambisonics recording of the cafeteria of the University of Oldenburg (Hendrikse et al. 2018a). In order to achieve a diffuse reproduction of the background sound field and to avoid spectral artifacts due to self-motion, the first-order signal was extended to 7 th order, and a frequency-dependent rotation similar to the method of Zotter et al. (2014) was applied. The average sound levels for each conversation were measured with a sound level meter at the position of the listener. The sound levels for the YNH were 45.2 ± 0.3 dB A for the conversations and 49.7 dB A for the cafeteria background noise. For the older participants, the speech levels had to be increased and the noise levels reduced, as the first two older participants complained that they could not hear the spoken instructions clearly inside the simulation (speech in quiet). The levels for the ONH were adjusted with an increase of 3.1 dB for speech (48.3 dB A) and a decrease of 3.7 dB for noise (46.0 dB A). The levels for the OHI were adjusted with an increase of 9 dB for speech (54.2 dB A) and a decrease of 6 dB for noise (43.7 dB A). These level adjustments were defined by the first participants (first ONH and first OHI). The speech level was raised so that they could understand the speech in quiet and the background noise level was reduced so that the conversation could be followed when background noise was present. We were aiming for a realistic speech level around 65 dB A, but due to a calibration error, the actual speech levels were not in this range.
Visual Stimuli. Three different visual conditions were presented in this experiment (see Fig. 1): audio-only (AO), virtual characters (VC) and video recordings (VID). In the CS-AO condition, the projection was turned off and a diffuse light was turned on. In the HMD-AO condition, a virtual laboratory was shown, so the participant would feel he/she was in the same real space and would have some reference points: the participant could see the chair underneath, the platform where the chair was, the cylindrical screen and the emergency button. This virtual scene was used for the other visual conditions. For the VC condition, the 3D virtual characters were created with Makehuman version 1.02 in resemblance to the real actors. The virtual characters were blinking and moving their lips with a speech-based lipsyncing (Llorach et al. 2016). The virtual characters also moved their head and eyes: they followed the conversation by looking towards virtual character who was speaking. These three animations were automated and generated in real-time. The effects of these animations can be found in the studies by Grimm et al. (2019b) and Hendrikse et al. (2018b). In the VID condition, the video recordings were shown through flat screens in the virtual scene (see Fig 1).

Experiment Procedure
The participants filled in an anonymization form and an informed consent. They were informed about the experiment through written forms, a video clip and orally. The interpupillary distance was measured with a ruler and the lenses of the HMD were adjusted accordingly. The head crown and the HMD were adjusted to the participant's comfort. If the participants used corrective glasses, we let them try the HMD with and without them; they decided whether they wanted to do the HMD trials with or without glasses. The EOG electrodes were attached to the participant together with the Bluetooth transmitter and participants were instructed not to touch them during the experiment. They were instructed that they would have to answer verbally 'A', 'B' or 'C', to the multiple-choice questions presented after each conversation.
After this introduction, they filled out the pre-exposure Simulator Sickness Questionnaire (SSQ; Kennedy et al. 1993) and were seated on the chair inside the tent. We included the SSQ in the experiment to assess whether participants suffered from cybersickness.
In order to calibrate the head crown facing towards the front, a cross was displayed on the CS. This cross showed where the head crown was pointing in the horizontal angle (yaw) in real-time.
Participants were asked to adjust the head crown on their heads until the cross was pointing in the same direction as their noses. The researcher helped and made sure that this was done correctly.
The participants started with the HMD or the curved screen randomly. They did the three randomized visual conditions with one display followed by three more with the other display. The order of the visual conditions was the same with the curved screen and the HMD for each participant. The conversations were randomized and each conversation was played equally often for each condition across all participants.
Instructions about the task were repeated again through a virtual character in the simulation. When using the HMD, an initial adaptation phase was added: a virtual character made suggestions for getting used to the room, to look at the chair they were sitting on and to find the emergency button behind them. If they did not find the emergency button, the researcher came inside the tent and made sure the participant could turn and see the button. The virtual button was in the same location as the physical one. This procedure was done to adapt the participants to the experience, e.g. some participants may be unaware that they can move or turn their heads with the HMD. This adaptation phase lasted around 1 minute.
After the instructions, there was a training trial. The training trial used a conversation that was not used in the test trials. After each conversation, the participants answered verbally to the multiple-choice related questions.
The participants came out of the tent to fill out the SSQ after all trials were completed. After this, we proceeded with the open interview recorded with a sound recorder.

Measures
Listening Behavior. In our experiment, the head yaw and the horizontal gaze angle were recorded. We computed three measures per trial meant to explain and characterize the listening behavior. Two measures were used to describe whether the head was turned towards the target talker. We computed the median of the head yaw when the target talker was at ±45-degree angle and at ±15-degree angle. Because the positions of the target talkers were symmetric, we flipped the sign of the head yaw when the target talker was at negative angles and used only two target talker angles (45-degree angle and 15-degree angle). These measures were named MedianHeadYaw45 and MedianHeadYaw15. To show and better illustrate what kind of values one would get per trial, we added two examples in Fig. 3. This figure shows how the sensor's data evolved during a trial. From these signals, three measures were computed per trial. In this particular example for the CS-AO trial (upper panel, with no visual), the MedianHea-dYaw45 was 0.3-degree angle and the MedianHeadYaw15 was 0.4-degree angle. In this trial, the participant looked ahead close to the 0-degree angle azimuth. For the HMD-VID trial (lower panel, video recordings with HMD), these values were 34.1degree angle for MedianHeadYaw45 and 15.4-degree angle for MedianHeadYaw15. In this trial, the head was turned towards the target talker. The third measure, the GazeDirError, was the RMS of the angle between the gaze and the target talker. The naming of this variable was kept as in Hendrikse et al. 2018b. This measure depicted whether the participants were looking at the target talker. In the example of Fig. 3, the values for the GazeDirError were 34.1-degree angle for the CS-AO trial (the participant was not looking at the target talker) and 11.9-degree angle for the HMD-VID trial (the participant was looking at the target talker). As there was no overlapping speech in the conversations, there was only one talker to attend to at a time. We omitted from evaluation 0.5 seconds before and 1.5 seconds after a talker change, as measured from the moment a new talker started talking.This was done to avoid movements related to switching attention (see Figure 4C in Hendrikse et al. (2018b) for more information).

Figure 3: Example of head yaw and horizontal gaze of the same participant in two different trials. On the top, data of a trial with the curved screen and audio-only (CS-AO) during conversation 2. On the bottom, data of a trial with the headmounted display and video recordings (HMD-VID) during conversation 4. Continuous black thick lines stand for head yaw, thin gray lines for gaze and blue circled lines for the angle of the target speaker.
Preference and Acceptance. The preference and acceptance of the audiovisual conditions were measured via a recorded interview. The participants were asked to give comments and impressions about the experiment once they completed all listening tasks. They were given a paper with six pictures (one for each condition) and a picture of each display device. We allowed a minimum of three minutes time and a maximum of 15 minutes for comments. Afterwards, the participants were asked to select one of the six conditions (see Figure 1) as the one they would like to experience in a future experiment. Then, they were asked to name the second best condition. Finally, they were asked to choose if there was any condition they would not like to experience again. The participants that did not have a preference between displays or visual conditions could also answer combinations, i.e., first preference as the video regardless of the display. The increase in SSQ symptoms between pre and post-exposure questionnaire was computed and the mean values for the total simulator sickness severity were below 13 for both groups. According to Kennedy et al. (1993), the cybersickness reported in this experiment is considered insignificant (10-15 Total Severity).

Listening Behavior
In this section, we analyze whether any of the factors (display type, visual condition and group type) affected the head yaw (MedianHeadYaw45, MedianHeadYaw15) or the gaze direction (GazeDirError). A mixed-design MANOVA (multivariate analysis of variance) was used for the analysis. The within-subject factors used for the analysis were the display type (CS and HMD) and the visual condition (AO, VC and VID). As a betweensubject factor, the group type (YNH, ONH and OHI) was used. The dependent variables were the MedianHeadYaw45, the Me-dianHeadYaw15 and the GazeDirError. To correct for multiple comparisons (Cramer et al. 2016), the false discovery rate was controlled with the Benjamini-Hochberg procedure (Benjamini and Hochberg 1995) Figure 4 (MedianHeadYaw45, MedianHeadYaw15) show that when there was no visual information (AO), participants did not turn their heads away from the frontal direction. When visual cues were available (VC, VID), participants turned their heads towards the target talker. Pairwise comparisons showed that this effect was significant for MedianHeadYaw45 between AO and VID (p< 0.001), and between AO and VC (p< 0.001), and for MedianHeadYaw15 between AO and VID (p< 0.001) and between AO and VC (p = 0.012). The right panel in Figure 4 (Ga-zeDirError) shows that when the target talkers were visible (VC, VID), the participants looked at them. This effect was significant, as pairwise comparisons showed that GazeDirError was significantly different between AO and VC (p< 0.001), and between AO and VID (p< 0.001). Additionally, it can be seen in the right panel of Figure 4 that the head yaw and gaze direction were closer to the target talker when using video recordings (VID) than with virtual characters (VC). Pairwise comparison found that this effect as significant for MedianHeadYaw45 and GazeDirError (p< 0.001), but not for MedianHeadYaw15. To a lesser extent, when the participants were wearing the HMD, their heads were turned closer to the target talker than with the CS when visual cues were available (VC, VID), as shown by the light grey and dark grey vertical histograms in Figure 4. This display type effect was only found significant for the MedianHeadYaw45 (F(1, 32) = 15.61, p = 0.002). No significant differences between display types were found for the MedianHeadYaw15 and GazeDirError. Group type did not show a main significant effect (F(6, 62) = 0.57, p = 0.75), meaning that the listening behavior was not affected by age or hearing type in this experiment. No significant interaction effects were found, therefore we can consider that the effects of display type and visual condition were independent from each other.

Preference and Acceptance
In this section, we present the results of the interviews. The two older groups (ONH and OHI) were grouped together in this section as we considered age, and not hearing type, as the important factor for technology acceptance.
Open Comments. We analyzed the recorded interviews and annotated the issues that were mentioned: these are summarized in Table 1. The interviews revealed that the speech was difficult to understand (Table 1.2); some subjects found the males talkers more difficult to understand (Table 1.3-5); some found the accent of the non-native female talkers hard to understand (Table 1.6). Three participants mentioned that moving their head changed their audio perception (Table 1.7). Six participants mentioned that the HMD was heavy and three older participants commented that they felt isolated when wearing the HMD (Table 1.8-9). Six YNH participants noticed that the screen of the HMD was brighter than the CS (Table 1.10). Seven participants mentioned that in the AO trials it was easier to concentrate than in the other trials, but for three participants it was the opposite (Table 1.12-13). Additionally, eight participants mentioned that it was easier to understand the conversation in the VID condition (Table 1.14). Six older participants complained about the insufficient resolution of the lips of the virtual characters (Table 1.16), four participants mentioned that the virtual characters were too stiff (Table 1.17) and seven participants indicated that the characters were not realistic (Table 1.15).
Chosen conditions.Participants were asked to select, out of the six possible conditions, the two most preferred, and to mention whether there was any they would not like to repeat. The partici-pants could also choose a visual condition, e.g., VID, regardless of the display or vice versa. The first four subjects were not asked whether there was any condition they would not like to do again. The answers of the participants are shown in Table 2. We divided the preference results by visual conditions and display. The first and second preferences were grouped together, e.g., the VID condition was chosen by thirty-four participants out of thirtyeight as the first and/or second preference. Eighteen participants out of thirty-four were willing to do all the conditions again.
The VID condition was clearly chosen as the preferred visual condition and was never rejected. The other two visual conditions, VC and AO, were chosen with nearly equal preference. The YNH participants showed no preference between the HMD and the CS displays. The older participants (ONH and OHI) preferred the CS more often (twenty participants chose the CS and fourteen chose the HMD as first/second condition out of twenty-one). In general, the HMD was more frequently rejected (by ten participants out of thirty-four) and the CS was rejected only by two participants out of thirty-four. The rejected conditions were always a combination of a display (HMD or CS) with the AO or VC condition. The AO condition was rejected by four participants more than the VC condition (twelve vs. eight participants out of thirty-four). Relatively more YNH participants (six out of thirteen) rejected the AO condition, compared to the older participants (five out of twenty-one).

Listening Behavior
Listening behavior was affected by the kind of display used. That being said, significant differences between displays were only found for the head orientation when the target talkers were on the sides. These differences were below 10-degree angle in average. We suspect that the participants were doing the same listening strategy, i.e., they were not turning their head to improve their listening situation, and that these differences were caused by the reduced field-of-view in the HMD: participants turned their heads few more degrees to the target talkers when wearing the HMD to see them clearly. This bias, which we suspect is different for other HMD models, should be taken into account when doing similar experiments with target talkers located away from the center.
As expected from the literature (Grange et al. 2018;Hendrikse et al. 2018b), visual cues affected listening behavior: participants tended to look straight ahead in the AO trials and they looked at the target talkers when visual cues were available. Significant differences between the video recordings and the virtual characters were found, as reported by Carter et al. (2013), and were rather small: for the head orientation, the difference between VID and VC was only significant for the side talkers and was around 5-degree angle; for the gaze behavior, this effect was less than a 5 degrees in angle, which was relatively small in comparison to the accuracy of the eye-tracking (approximately ±10-degree angle). These differences were not significant in our previous study (Hendrikse et al. 2018b). One possible explanation for this small effect being significant in our study is that we measured more participants.
Even though we found significant differences between display types and visual conditions, the effect size was rather small when comparing to the audio-only visual condition. Including visual cues in an audiovisual environment with surrounding talkers will change drastically listening behavior. We find it crucial to include visual cues in an experiment that targets higher ecological validity. The kind of audiovisual technology used will introduce small bias that should be considered in specific cases where head orientation and gaze are crucial.
Because the SRN was different between the group types, no final conclusions can be drawn regarding group differences in listening strategies, as it has been found that the SNR can affect listening behavior (Hadley et al. 2021). Further research should look into group differences, as audiological evaluations in the clinic are mostly targeted to older and hearing impaired individuals.

Preference and Acceptance
As expected, based on the study by Philpot et al. (2017), the YNH participants showed equal preference for the two displays. The older participants preferred the CS over the HMD. The HMD was rejected more often as a display and also received more negative comments, such as that it was heavy, isolating and distracting. Therefore, the CS would be a better choice for the comfort of the participants. Nevertheless, the HMD was chosen quite often as a first or second option and it was never rejected as a display alone, i.e., regardless of the visual condition. Therefore it should be considered for clinical implementations, as a cheaper and simpler implementation.
The video recording condition (VID) was clearly the most preferred visual condition. This finding was in agreement with the previous study of Hendrikse et al. (2018b). The AO condition was the most rejected (by eleven participants out of thirty-four), showing a general preference for conditions with visual cues. The comments regarding the VC condition indicated that their quality, non-verbal behaviors and lip-readability should be improved. It is worth noticing that only the older participants (two ONH and four OHI participants out of twenty-one older participants) mentioned the lip-readability, indicating that older participants might look for this kind of visual cues specifically.
The accent was brought up as a difficulty for understanding, but the female talkers, which were the ones with the accent, were also specified to be easier to understand by other participants. We consider that the talkers were all more or less equally intelligible, according to the open comments.

Outlook and Limitations
Depending on the research question, current hearing clinics and laboratories might want to use immersive visual cues (Keidser et al. 2020). Changing their methodologies from audio-only to audiovisual stimuli might be expensive and effortful. HMDs are more affordable and easier to setup than custom-built CSs. Nevertheless, specific procedures need to be done in the clinic for HMDs, such as measuring the interpupillary distance and adjust-ing the head-straps. Which audiovisual system to use will depend on each specific experiment and clinical setup.
Next generations of HMDs might improve some of the issues mentioned by the participants, such as the weight of the device. Mixed reality and augmented reality solutions should also be considered, as they might be less isolating than HMDs. Further improvements need to be done to our virtual characters if they are to be used (see Llorach et al. 2018). In this study we used open source characters and animations available at the time. This is supported by the comments of the participants and the significant differences found between the video recordings and the virtual characters.
In the current experiment, the task of the participants was to listen to real conversations and answer content-related questions. Nevertheless, in realistic noisy situations, the talkers would modify their voices to be clearer and louder (Lombard speech), but in this experiment, the speech stimuli were recorded without background noise. This fact and the low reproduction levels increased the difficulty of the task. Many participants noted that it was hard to understand and/or difficult to concentrate. Whether the behavior would be different in less challenging situations cannot be derived from this experiment. For further information on listening strategies with different SNRs, please refer to Grange et al. The ecological validity of this study is limited to tests done in the laboratory. Whether the participants move in the same way in the field and in real-life situations cannot be derived from this experiment. Additionally, social presence, interaction, and even culture (Endrass et al. 2009) might change the listening behavior. Wearing glasses should be accounted for in future experiments of this type, as they might change the head and gaze relationship. Future research should take into account these factors to better understand the listening behavior.

CONCLUSIONS
Visual cues are needed in experiments where the head yaw and gaze direction in relation to the sound source position are relevant. In this type of experiment, the head-mounted display (HMD) makes participants turn the head more towards the target talker when it is further away from the center in comparison to the curved screen (CS). The video recordings (VID) have the same effect and additionally make participants look more at the target talker in comparison to the virtual characters (VC). The effects of display (HMD and CS) and type of visual cue (VID and VC) on head yaw and gaze direction were relatively small, but they should be taken into account for audiological experiments and testing of hearing aids where these measures are crucial. These effects were bigger when the target talkers were further away from the frontal direction.
Overall, the CS and the video recordings were the preferred conditions. Nevertheless, HMDs and virtual characters can in principle be used to evaluate head yaw and gaze direction.