{'fraction_non_alphanumeric': 0.02546710242338287, 'fraction_numerical': 0.0067213418018129125, 'mean_word_length': 4.888707334785766, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Evaluating the Impact of Highly Immersive Technologies and Natural Interaction on Player Engagement and Flow Experience in Games

. Social interaction in videogames has a big impact on players experience and is often used to increase enjoyment and retention. In the current study a highly immersive setup based on the Oculus Rift and depth cameras and exploiting natural user interaction is compared with a classical Keyboard & Mouse conﬁguration in the context of a videogame experience taking place in a shared Virtual Environment. The research aims at assessing the impact of new technologies and interaction metaphors on users engagement when playing social games. Initial ﬁndings from our study suggest that while players perform better using the classic Keyboard & Mouse setup, the new technological setup and the Natural User Interface oﬀer higher level of engagement and facilitate user ﬂow state.


INTRODUCTION
The importance of the social component as motivation for playing has been pointed out by many researches [18,3], both in digital and non-digital games. When playing with other people involvement as well as enjoyment of the game increase. Lazarro describes in her work [10] the main reasons that brings people to play games identifying four keys, or pathways, leading players to emotion in games. Social interaction is one of these four aspects. Yee [19,18] offers a clear slice of the characteristics and behaviours of Massively Multiplayer Online Role-Playing Games (MMORPGs) players. He analysed the motivation of over 30000 MMORPG users, identifying three main components that define the motivation to play online games: achievement, immersion and, again, the social component.
Social interaction in videogames is mostly defined by the ability of players to communicate with the others. The less is the effort required in order to learn how to interact with the others, the better is the experience gamers will have. By providing additional ways to interact in games, players can customize their experience, choosing the most familiar and effective forms of communication for them. The social component is therefore heavily influenced by the available technologies. Nowadays the hardware evolution has lead to powerful solutions able to substantially improve the interaction of players with the Virtual Environment(VE). The spreading of depth cameras and sensorized controllers is shaping the way we play. Natural User Interfaces 3 (NUIs) are becoming more and more popular. New richer interaction metaphors can be designed in order to improve the game engagement in a social scenario. With the availability of cheap highly immersive visualization systems (e.g. the Oculus Rift), a completely new experience can be provided to the players.
The presented work aims at assessing the impact on player engagement and social presence of new highly immersive technologies combined with NUIs. During the experiment subjects have played a collaborative jigsaw puzzle game in a shared VE using two different interaction metaphors mapped on two different technological setups. The first interface exploits Keyboard & Mouse, one of the most traditional gaming interface, as a medium between the player and the VE, while the other uses a Head Mounted Display(HMD) and a depth camera implementing a NUI.
The paper is structured as follows: first a review of previous works addressing player engagement and social presence is presented. Then the study methodology is explained: the technological setup and NUI used to conduct the experiment, the participants sample, the procedure and metrics are introduced. Finally the salient results are discussed and guidelines for the development of future games are provided.

LITERATURE REVIEW
If players do not enjoy the game, they will not play the game [16], therefore player enjoyment is the most important goal for computer games. There are multiple factors contributing to the overall game engagement [15,7,1]; flow and social interaction are two of them. Flow has been defined by Csikszentmihalyi as a state of mind in which a person is completely involved and immersed in an activity [4,2]. The concept of flow is central to game evaluation [12,16,7,1].
Social interaction represents another key aspect when designing videogames. People often play games to interact with others, regardless of the task [16]. Players use games as mechanisms for social experiences: "It's the people that are addictive not the game" [10]. Several researches investigated how the social component in videogames affects user engagement and satisfaction [5,18].
Given the relevance of these two factors on gaming experience, it is extremely important to maximize their effects when designing a game.
Social interaction in videogames is mostly defined by the ability of players to communicate with the others and, more than other aspects in videogames, it is heavily affected by the technological facilities used. This difference is even more important when players are not physically co-located and the communication among them relies only on the technological layer. Using different hardware solutions new interaction metaphors can be enabled: motor activity-centered games exploit new console controllers (e.g. Wii Remote); depth sensors allow full body interaction. Game designers can take advantage of them to create new communication channels or improve the existing ones.
Gajadhar et al. [6] evaluated the effects of co-player presence on player enjoyment according to three common two-player settings (virtual, mediated, and co-located). They used a basic technological setup in which subjects play PONG varying the closeness of the players. They found that players enjoy more the co-located setting due to the increased affordance for communication. Nowadays the increasing availability of novel hardware devices (e.g. Depth cameras, HMDs, inertial sensors) provides new and interesting alternatives to game designers. It is possible to develop novel, powerful and extremely immersive social experiences overcoming the existing communication gap between co-located and remote players. It is today possible to de-materialize the players and teleport them in a shared virtual world where the game takes place.
Sajjadi et al. [14] investigated whether the choice of interaction mode\controller has an impact on the game experience. They tested a collaborative game using the Oculus Rift and Sifteo Cube 4 . They didn't found any significant difference between the two interfaces on the game experience. They instead observed that almost all participants using the Oculus Rift looked for alternative way of communication trying to use gestures to interact with the partner even if not enabled by the technological setup.
Lindley et al. [11] focus on the impact of the new interfaces involving body movements on player engagement and social behaviour. They found that the amount of social interaction is higher when using input devices which allow body movements, resulting in an higher engagement in the game.
Kauko and Häkkilä [8] compared the effect of two different technological setups on social interaction. Subjects played the same multi-player game first on their mobile phones facing each other and then on a typical game-console setting side-by-side. They found an increase in the social interaction in the first setup which enables a socially richer game experience.
Even if both flow and social factors contribute to an increment of game engagement, it is still not so clear which kind of interactions occur between them. Sweetser and Wyeth [16] assert that social interaction, being not an element of flow, can interrupt immersion in games, as real people provide a link to the real world that can knock players out of their fantasy game worlds. Similarly Lindley et al. [11] suppose that by encouraging social interaction, players will in some sense have been drawn out of the game environment and into the real world breaking the flow.  All the previous works highlight the importance of communication between gamers in multiplayer videogames. Being able to effectively interact with your partner is extremely important. At the same time the social communication works as a link with the real world because often happens outside of the game. This can affect the flow experience which represents another fundamental aspect for game enjoyment. New technologies offer powerful alternatives to enable new ways of communication. In this study we evaluated a new NUI designed in order to fuse all the required communication channels inside the game world. The present experiment aims at assessing if the adoption of new technologies can satisfy the needs for communication richness in social games.

METHOD
The current work aims at assessing the impact of new highly immersive technologies combined with natural interaction on player engagement and social presence. Particular attention has been directed to the relation between flow experience and social interaction. The subjects participating in the study play a collaborative jigsaw puzzle game in a shared VE using two different interaction metaphors tied to different visualization systems.
In the following, first the technological setup and interaction metaphors under evaluation are presented. Then the game used to perform the study is described. Finally participants, procedure and metrics are presented.

Technological setup and interaction metaphors
Two identical networked hardware setups have been used to play the collaborative game. Each system is composed by: (1) a gaming-grade workstation; (2) an Oculus Rift DK2 HMD; (3) a Primesense Carmine 1.09 RGB-depth (RGBD) camera; (4) a wireless headset with microphone; (5) two coloured thimbles.
The first interface proposed exploits Keyboard & Mouse, one of the most classic gaming interface, as a medium between the player and the VE. Players can navigate the environment by using the mouse to change the view direction and the keyboard to move their point of view. They can grab and position the puzzle tiles and zoom in and out by using the mouse. Verbal communication is enabled by using headphones and microphones. Each player can see both his and the partner's pointers (see Fig. 1). We refer to this setup as "KM" from here on.
The second interface provides a natural interaction between the player and the VE using an implementation of [17]. The user wearing the HMD is free to move naturally in a physical\virtual environment of about 3 by 3 meters. User head's tracking is performed by using the inertial measurement unit built-in the Oculus and the tracking camera shipped with it. The perception of the self and the partner as well as the natural interaction with the VE are enabled by depth cameras and inertial\visual tracking techniques. Two coloured thimbles placed on the index and thumb of the dominant hand are tracked using the acquired RGBD data. Through a simple collision detection algorithm applied to the positions of the thimbles, the user is able to naturally interact with the virtual objects by grabbing and moving them in his\her peripersonal space (see Fig. 2). The participants are able to communicate both verbally and using their bodies (e.g. using gestures). No haptic feedback is provided. We refer to this setup as "OU" from here on.

The game
A collaborative jigsaw puzzle game has been developed for the experiment. This popular game genre combines a low complexity with an high level of attention and interaction. Even if videogames of this type are usually two-dimensional, we designed the game to be played in a three-dimensional environment in order to exploit the immersive capabilities of the HMD. The collaborative component of the game, based on the Complementarity and Shared Goals design patterns [13], allows to evaluate the effect of social interaction between participants on the game engagement and flow.
The game scene is composed by a virtual room with a table on a side. A countdown timer, the scoring board and a poster showing the solution are hung on a wall. The puzzle is made up of 48 tiles randomly disposed on the two sides of the table. Each tile is represented by a gray parallelepiped with the top face textured with a part of the puzzle image. In the middle of the table a board divided in two sections defines the placeholders where the puzzle has to be arranged on (see Fig. 1 and Fig. 2).
The game is played at the same time by two players physically located in two different rooms but sharing the same VE. The actions performed by a player to the environment are visible to the partner (e.g. scoring, tiles movements and positioning). During KM sessions the players can see his/her own and partner's mouse pointers. During OU sessions the RGBD captures of the bodies are streamed between the two setups. A proxy for each player, made by a textured mesh reconstructed from the RGBD data and a virtual head replicating the user movements, is shown in the VE (see Fig. 2).
Both participants can interact with each tile at any time. When a tile is currently grabbed by a user, the other player can not interact with it until it is released. A tile dropped close to a free board placeholder is attracted and automatically positioned on top of it. If correctly positioned, the tile collapse on the placeholder and cannot be moved anymore. Sound feedbacks notify correct or wrong tile positioning. Each player is able to place tiles only on the half board belonging to him/her. Trying to position a tile on the half board belonging to the partner, causes the tile to jump away in a random position. Each player can see only the half solution owned by the partner so that players need to help each other. The two players are characterized by different colors, red and blue. Each element belonging to a user (score, grabbed tile, etc) is modulated with the correspondent color.
The aim of the collaborative game consists in working together with the partner in order to solve the puzzle before the time is over. Players have seven minutes to correctly place all the tiles on the board.
The shared team score is calculated according to the following rules: • +2 points for each correctly positioned tile • +1 point for each 5 seconds left when the puzzle is completed • −1 point for each tile not correctly positioned when the time is over

Participants
The participants have been recruited among colleagues and students. A total of 24 subjects, 15 males and 9 females healthy subjects, aged between 23 and 50 (32, 04±6, 84) took part at the experiment. Only 2 of them were not native Italian speakers. During the recruitment they have been asked to read and sign the informed consent. Thus they filled an entry questionnaire (EnQ) used to collect demographic information like gender, age and level of education. In the EnQ users had also to rate, on a 5 points Likert Scale from 0 to 4, their experience with the use of computers (average 2, 88 ± 0, 85), videogames (average 2, 12 ± 1, 33), use of immersive virtual displays(average 1, 54 ± 1, 21), puzzle games (average 2, 04 ± 1, 2) and online puzzle games (average 0, 83 ± 1, 05).

Procedure
During the recruitment, participants have been asked to play a single player version of the puzzle game using keyboard and mouse as input devices. In a social game the challenge is heavily influenced by the different abilities of the participants [2]. Owning similar skills is extremely important in order to make the game challenging for both players and prevent boredom or frustration. For this reason a pre-experiment aimed at assessing the puzzle-solving abilities has been conducted in order to couple participants according to their dexterity. The subjects were unaware of the real objective of this session. Hence twelve couples have been formed. Before the experiment, each couple has been informed about the outline of the experimental session. The players have been then divided on the two identical setups (see Sec. 3.1) prepared for the experiment, located in two different rooms and network connected. The subjects, spatially not co-located, have been able to communicate by using only the communication channels provided by KM or OU session. In addition, each user performed a 5 minutes trial session to get familiar with the NUI provided playing a simplified (12 tiles) single player version of the puzzle game. Two different puzzle images have been used.
The experiment adopted a within-subjects design. Each couple played two sessions, one for each interaction metaphor, KM and OU. Players were allowed to read again the instructions before each game. The order in which the two game sessions as well as the puzzle images were presented to different couples has been randomized.

Metrics
In order to analyse the player engagement, we have reviewed available questionnaires focusing on user engagement and social experience. Qin et al. [12] focus too much on the game narrative aspects while Seif El-Nasr et al. [15] in their metric do not consider player flow experience. GameFlow [16] represents the baseline from which both Immersion Experience Questionnaire(IEQ) [7] and Game Engagement Questionnaire (GEQ) [1] have been developed and it is outdated by these two research works. Analysing the questions composing IEQ and GEQ, we found that even if the two questionnaires namely address different factors, immersion and engagement, they are very similar. We therefore decided to adopt GEQ which is completed by the social questionnaire named Social Presence in Gaming Questionnaire (SPGQ) [9], addressing another fundamental aspect of our research.
At the end of each game session, players have answered a post condition questionnaire (PCQ) composed by a subset of the GEQ items (competence, flow, tension\annoyance, challenge, negative affect and positive affect), a subset of the SPGQ items (empathy and behavioural involvement), awareness and satisfaction questions. When the two conditions have been played, an exit questionnaire (ExQ) has been presented to both players in order to collect their preferences and motivations, friend relationship, general impressions and suggestions. Finally an informal debriefing session between the experimenters and both players has been conducted to further register impressions and anecdotes.
Besides data collected through questionnaires and interviews, objective measurements have been recorded through the game in both the preliminary and the experimental sessions. Usage and performance data collected comprehend: (1) completion time and score, (2) frame-rate and network latency, (3) outcome and tiles positions, (4) positions and headings of player head. Each session was video and audio recorded for further investigations. Experimenters assisted all the sessions taking notes of noteworthy events.

RESULTS
The questionnaires results have been analysed in order to assess the impact of the different technological setups and interaction metaphors on user engagement, social presence, awareness and performances. Following the instructions provided by the authors in [1], the answers to GEQ and SPGQ questionnaires have been aggregated in order to obtain a value for each one of the eight items taken from the two questionnaires. A Wilcoxon signed-rank test has been used to statistically compare questionnaires results for the two conditions as the distribution of the data was not Gaussian. A paired t-test has been used to compare performances.

Awareness, Satisfaction and Performances
As reported in Fig. 3, players had a good awareness of the other's actions, locations and intentions in both setups. All the participants rated both experiences as very satisfying as shown by the question "Please rate your overall satisfaction" reporting an high score in both sessions. Answers to the ExQ showed a clear preference of the participants for the OU session. To the question "Which kind of user interface do you prefer?", 16 players (∼ 66.7%) answered the natural one. Figure 3 shows the performances registered during the game sessions. Data reports a significant difference between the two setups in terms of frequency of tiles correctly (W = 8.00, p = 0.015) and wrongly (W = 11.00, p = 0.028) positioned.

Game Engagement and Social Presence
The results of GEQ questionnaire indicate an overall positive evaluation of both game setups (see Fig. 3). Participants felt competent in both sessions without any particular difference. Even if in both setups players have reported a high level of flow, the psychological absorption has been significantly greater in the OU session (see Fig. 3). The single questions of the GEQ flow item (see Fig. 3), (b) "I forgot everything around me" (W = 4.00, p = 0.048), (d) "I was deeply concentrated in the game" (W = 5.00, p = 0.013) and (e) "I lost connection with the outside world " (W = 6.00, p = 0.046), highlight the main differences characterizing the OU experience.
Challenge has been medium-rated by participants. Players found the OU session significantly more challenging with respect to KM (W = 22.5, p = 0.011). Low values for tension/annoyance and negative affects have been highlighted for both the sessions. A slightly higher value of annoyance, (a) "I felt annoyed " (W = 0.0, p = 0.023), have been registered when playing with the Oculus. Players reported a high positive affects in both sessions.
Participants high-rated both social components, Empathy and Behavioural Involvement. No relevant differences has been found in the results.

DISCUSSION & FUTURE WORK
In the present work we have explored the effect of two different interaction metaphors and technological setups on game experience taking place in a shared Virtual Environment. A number of design implications and trade-off can be deduced starting from the findings of this study.
Almost all the players enjoyed the OU metaphor and most of them (66.7%) found it preferable to the classic Keyboard & Mouse interface even if it resulted to be more challenging. The playing experience with the Oculus has been perceived as more engaging and entertaining. Almost all the participants who preferred the KM metaphor appreciated the lower complexity of the interface, which results more familiar and comfortable for people who daily use computers. They were able to become proficient in a shorter period, while the OU configuration required more time to get comfortable with. Most participants (71%) during the final debriefing asserted to have appreciated the natural interaction with the environment and the partner because it makes the experience more similar to the reality ("I felt like I was really playing with him a real puzzle!").
In the Keyboard & Mouse setup users mostly use the mouse to point tiles to be grabbed and locations where to put them on. While the mouse indication is the preferred method, verbal communication was nonetheless extremely important to support it. Players used to describe the graphic features of the tiles to grab (e.g. "Take the big red bird!") and sometimes to number the destination placeholder (e.g. "Put it on the third column, four cells up."). In the Oculus setup, participants extensively used all the available communication channels provided in order to complete the task. Experimenters observed that most of the players preferred using hands gesture to interact with the other (see Fig. 2): "Wow, I can point out them! ". Verbal communication was nonetheless extremely important to support gestures. Also in this case many participants described the tiles to grab or the action to undertake verbally in addition to using their hands.
All the subjects reported that the NUI was more challenging with respect to the KM metaphor, however only one player found the OU metaphor not enjoyable and too complex to be used. The majority of players (75%) have considered the Keyboard & Mouse more immediate and faster. Nonetheless, during the KM session, experimenters observed that many players tried to use body language and gestures to interact with the other, even if these communication channels were not available. The same attempt to communicate with the partner using gestures has been reported also in [14]. Hence providing a natural interaction seems to be important during social activities. If the NUI would be completely transparent, removing any artificial medium between the user and the social sphere, probably it would be perceived easier and more enjoyable than the mediated communication (e.g. Keyboard & Mouse).
The game environment has been designed in order to maximize the space needed by a player during a game session and stimulate participant's movement to evaluate the spatial awareness in a social shared environment. In the OU setup, due to the nature of the technology and the choices made, players were able to see just a part of the scene at once and therefore were forced to walk and rotate the head in order to play. On the contrary, when playing using the KM setup, players had a global view of the entire scene, without being required to move. Even if it was possible to change the point of view using Keyboard and Mouse, nobody did it. This aspect may have affected the interfaces comparison in terms of usability. Adopting a different game design forcing the players to move and rotate the view in the KM as in the Oculus setups, would probably generate different results. It would be interesting to evaluate this different conditions.
Thanks to the high immersion and sense of presence induced by the technology, participants during OU session perceived the proxy of the other more as a physical presence rather than a virtual representation. This makes the experience more engaging. At the same time, the absence of any physical feedback and the possibility to pass through the representation of the partner has been perceived by some players odd and sometimes a bit annoying for the purpose of the game, while cheerful by others. The essential KM interface resulted to be more functional to the task but, as observed by the experimenter and highlighted in the open questions, less funny and more impersonal. We plan to explore possible solutions addressing "virtual proxy compenetration" in dedicated experiments where different stimuli and virtual warnings will be provided to avoid proxy collisions (e.g. using a "virtual aura" to signal proximity) or to reduce the related visual impact (e.g making the representation of the other player transparent as it comes closer).
Differently from what presumed, there was no significant differences between the two configurations concerning the awareness of the partner. Even if the OU metaphor provides more detailed and richer information on what the other user is doing, players were able to equally distinguish other's intentions and actions in the two setups. This is mainly justified by the heavy usage of verbal communication to coordinate the team actions. Partner location awareness obtained similar results. Players focusing on the puzzle completion do not require rich and accurate information about partner location and consider equally satisfying the two modalities.
Almost all the participants were Italian native speakers with two exceptions. Experimenters noticed that players speaking different languages benefited more from the NUI. Language misunderstandings were compensated by gestures. Due to the small number of non Italian native speaker it has not been possible to evaluate the real impact on the user engagement. It would be interesting to further investigate the impact of linguistic differences on player engagement.
The sample used in the experiment was composed by people who daily use computers. All the participants were at ease with the use of Keyboard & Mouse. It would be interesting to evaluate the impact of the two metaphors on a more variegated sample to highlight possible differences related to previous personal skills.
In this work a wider communication interface has been presented in comparison with a traditional Keyboard & Mouse setup. Both the interaction metaphors have been tested in a social collaborative shared environment. Flow experience is one of the key factor to make a game engaging as like as the social component. Players completely engrossed in a game reach the flow state. As a consequence of the deep absorption, as described by Jennett et al. [7] in their work, being increasingly immersed in a game decreases one's ability to re-engage with the "real world". During the debriefing session, a player talking about the OU experience said: "The interruption has a much stronger impact; the break is much clearer ". The study results show a significant increment in the participants flow experience in the OU configuration. The deep immersion provided by the technological solution together with the natural interaction have led to a greater absorption in the game. Sweetser and Wyeth [16] speculate on the effect of the social and flow components in games. They assert that reaching the flow mental state is impeded by the social activity which establish a link between the player and the real world. In the OU setup using the NUI, being the social interaction fused into the virtual environment, players do not need anymore to "leave" the game in order to interact. The link with the real world requested by the social communication is therefore broken. In order to investigate this effect, it would be interesting to develop a single-player version of the game using the same immersive setup and the NUI. If the provided communication channels would be expressive and transparent enough to make the virtual experience seeming real, the comparison with the multi-player game should not highlight any relevant difference in terms of flow experience. On the contrary, it would be possible to observe a significant increment in the flow due to the greater engagement reachable with the social component.

CONCLUSION
Based on the results, we can conclude that the players had an overall good game experience with the developed interface providing natural interaction, which proved to be an interesting alternative to the classic Keyboard & Mouse interface. The technological setup composed by the Oculus Rift combined with the NUI can be used to increase game engagement. No significant differences have been observed on the social presence between the two setups and both of them were perceived equally positive.
When designing social games a broad communication bandwidth enriches players gaming experience and increases enjoyment. A natural interaction with the environment, like the one provided in the study, allows the users to experience ways of communication similar to what they use in real-life. This turns out to be extremely important to support the richer interaction happening during collaborative tasks.
Thanks to the new technologies, nowadays it is possible to de-materialize the players and teleport them in a shared virtual world where the game takes place. Games can be played without the medium of any interface but using the own body as a way to interact with the VE.
Even if the new natural interaction has been described as more intriguing and enjoying, the classic setup based on Keyboard & Mouse resulted to be a better choice when performances matter. The study highlights that the implemented natural interaction results more challenging and requires more time to become productive. Nonetheless it will be interesting to evaluate if the performance gap can be reduced, or even nullified, by continuous training.
The OU settings have reportedly generated a significantly higher level of flow over the KM. This is mainly related to the natural interaction which enables new communication channels making the experience more real (a player said: "It was like playing a real puzzle"). Previous works [16,11] suppose a negative influence of social interaction on the flow experience. Encouraging sociability, players are in some sense drawn out of the game environment, back into the real world, breaking the flow. We suppose that the negative influence could be overcome providing natural communication channels embedded in the game world.
Looking at the current and upcoming available technologies, immersive social environments exploiting Natural User Interfaces have the potential to improve players game engagement and enjoyment defining the next gaming experience.