{'fraction_non_alphanumeric': 0.025952314612108427, 'fraction_numerical': 0.0066074911435736175, 'mean_word_length': 4.959203036053131, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Zoo Therapy for Alzheimer’s Disease with Real-Time Speech Instruction and Neurofeedback System

There is an increasing number of people with Alzheimer’s disease. Negative emotions are not only one of the symptoms of AD, but also the accelerator of the disease. Animal therapy can have a positive impact on the negative emotions of patients, but it has strict requirements for both environments and animals. In this study, we aim to explore the effectiveness of using virtual animals and their impact on the reduction of patients’ negative emotions to improve the user’s cognitive functions. This approach has been implemented in the Zoo Therapy project, which presents an immersive 3D virtual reality animal environment, where the impact on the patient’s emotion is measured in real-time by using electroencephalography (EEG). In addition to creating highly realistic virtual animals, the innovation of Zoo Therapy is also in its communication mechanism as it implements bidirectional human-computer interaction supported by 3 interaction methods: 3D buttons, speech instruction, and Neurofeedback. Patients can actively interact with virtual animals through 3D buttons or speech instructions. The Neurofeedback system will guide the animal to actively interact with the patients according to their real-time emotional changes to reduce their negative emotions. Experiments and preliminary results show that it is possible to interact with virtual animals in Zoo Therapy, and the Neurofeedback system can intervene in Zoo VR environment when the emotional value goes down and might reduce patients’ negative emotions.


Introduction
Alzheimer's disease (AD) is a chronic progressive neurodegenerative disease. In the progression of AD, when the gradual deterioration of cognitive function appears, emotional changes include emotional agitation, anxiety, irritability, depression, and social withdrawal also follow the entire process [1,2]. Such emotional fluctuations, with negative emotions, can cause distress to AD patients, their relatives and friends, and even medical staff. This leads to more negative emotions, a vicious circle. These negative emotions will in turn affect and destroy the cognitive function area of the patient's brain and push the further deterioration of the disease [3,4,5]. Due to the side effects of pharmacological treatment [6,7,8], non-pharmacological therapy has attracted attention. Among them, some stimulation-oriented treatments are designed to reduce the negative emotions of users have shown positive effects in studies [9,10,11].
Animal-assisted therapy (AAT) is also a stimulation-oriented treatment that uses animals as the treatment medium is different from therapies such as objects, music, and sports. Animals are living organisms. They have emotions, can perceive human behavior and emotions, and can interact with people. This human-animal interaction constitutes a strong emotional background. Non-organisms cannot give this kind of emotional feedback. Many studies [13,14] have shown the AAT can have a positive impact on the social behavior, the negative emotions, mental state, social interaction ability and quality of life of the AD patients. However, the various limitations such as maintenance AAT team [14,15], cost, and some ethical concerns [16,17] of AAT prevent this therapy from being widely promoted. Using virtual animals instead of real animals to participate in the treatment is a solution.
Although some studies [11,18] have tried to use virtual animals to complete AAT, there are still many shortcomings, especially how to perform the most important interactive part of AAT. Existing virtual animal treatments usually use cartoon animals, and their interaction is done by clicking on the interactive patterns on the user interface. It is not a direct interaction between the user and the animal. It is more like that a user interacts with a computer, rather than an animal. And these virtual animals cannot give emotional feedback because it cannot perceive users. The connection between participants and virtual animals is in one direction. All the emotional changes of the participants do not affect the actions and behaviors of the virtual animals.
As an overall improvement, also our contributions, we propose a brand-new virtual animal interaction therapy environment called Zoo Therapy. 1) It uses 360 degrees VR technology to allow users to interact with virtual animals immersivity. 2) Under the combined action of the three interactive methods 3D buttons, discourse (speech recognition system), and the user's emotions (Neurofeedback system), Zoo Therapy realizes the bidirectional interaction, to maximize the emotional feedback between the user and the virtual animal to ensure timely reduce the negative emotions of users. 3) we are the first to use the reinforcement learning-based method to autonomously learn the reaction rules of humans' EEG emotions in specific environments.
Our research questions are: (1) Is it possible to interact by the user's voice in a virtual reality environment? (2) The Neurofeedback system can send corresponding instructions to the virtual reality environment in real-time to intervene on the user's emotions. (3) Is it possible to reduce the user's negative emotions by interacting with virtual animals?
The rest of this paper is organized as follows. In section 2, we present our animal therapy and detail the different modules of the therapy that we developed. In section 3 we detail the experimental procedure. In section 4 we discuss the obtained results. Finally, in section 5, conclusion.

Our Approach: Zoo VR Therapy System
Our approach consists of a Zoo VR Therapy System which has realistic virtual horses and virtual dogs to participate in the interaction and supports three different interaction methods. The three interaction methods can be divided into two parts: subjective interaction and non-subjective interaction. Subjective interaction refers to the interactive instructions actively issued by the user and includes 3D Buttons and a Speech Instruction system. Non-subjective interaction is the interaction that the Neurofeedback system that guides the animal to actively interact with the user based on the user's real-time emotions. Neurofeedback system's EEG measuring module measures users' emotional reactions to the environment and the intelligent agent receives users' emotions and Zoo environment's information in real-time and intervenes in the Zoo environment by commanding animals depending on the emotions of the users and information of the environment. As shown in Figure 1.

Zoo VR Environment
"VR Zoo" creates a safe and economical virtual interactive environment including virtual animals. In this environment, the user can call the animal to come, eat, walk, stand up or ask it to leave at any time. Animals respond immediately when they receive instructions. Interactive instructions can come from three interactive methods. Our environment can be divided into five Functional Modules, namely, scene module, animal module, sound effect module, map module and human-computer interaction module. Following is a description for each module.
Scene Module -The overall appearance of the Zoo environment. In the scene module, we created a 3D treatment room (shown as Figure 2).
Animal models -Animals in the environment. The most common forms of AAT are dogs and horses. Therefore, we created a 3D treatment horse and a 3D treatment dog. (Figure 3 is the 3D horse in the treatment room). To complete the interaction between the virtual animals and the users, we also created an animation system when creating the animals to realize the animals' walking, eating, standing and other actions.  Sound effect module -Environment and animal sound effects. We do not only play soothing background music in the entire 3D environment. We also add different animal sound effects. For example, horse and dog can make several different calls in response to different commands from users, and the sounds of horse walking and eating, the dog's panting. The sound effect will accompany the animal's movement and change in realtime according to the different actions of the animals.
Map Module -The trajectory of animal movement and the generation of interactive routes. The function of the map module is to calculate and generate a feasible path according to the animal's real-time position and state, so that the animal will update its new path depending on the user's new instructions and approach or go away from the user.
Human-computer interaction module -User interaction with the environment. First, the user can interact with the animals by selecting the 3D button in the environment. Then, after completing the model training of the speech recognition system, we will directly add the speech recognition system. Users can talk to the virtual animals to make them come, walk, eat, or leave. In addition, a Neurofeedback system has been also added to the environment to influence animal behavior by identifying emotional changes in the user's EEG. When the three instruction modules work together, we give priority to the 3D button and speech recognition system. We will introduce the speech instruction system in section 2.2 and the Neurofeedback system in 2.3.
Finally, we integrate the above five modules to form the final VR Zoo Therapy.

Speech instruction system
We added the speech recognition system that allows users to interact with animals by using their voices. We directly selected an End-to-End ASR (Transformer [21]) model [22] which accepts a spectrogram parsed from the user's voices as the input and the output is text. This system's goal is to input the user's voice, recognize the text content of the voice through our trained speech recognition model, and determine whether the user has issued an instruction to the animals based on the content of the text. If there are instructions, the model will send them to the environment, and the animals in the environment will complete the task according to the instructions. The horse in Zoo therapy can receive five commands: come, eat, walk, stand up, and leave. Therefore, the speech recognition model is particularly sensitive to these words. During the use of the Zoo environment, which is synchronized to 16 kHz sampling rate recorded audio. A waveform audio file is generated every three seconds, and then passed to the trained ASR model in turn for further processing. Once the ASR model receives audio file, it will parse it to a spectrogram, and then predict the audio content. The model converts it into text information. Finally, the predicted audio content will be judged whether it contains the above five instructions. If so, the model will send instructions to the Zoo environment and execute them.

Neurofeedback System
The Neurofeedback System monitors the user's emotional state in real-time and allows the animals to respond to the user's emotions. Our Neurofeedback system includes EEG measures and an RL-based agent, between which EEG measures are responsible for monitoring the user's EEG and passing it to the intelligent agent. The agent will automatically determine whether to allow the animal to actively interact with the user without the user giving an instruction to the animal based on the user's EEG information, thereby reducing the user's negative emotions.

EEG Measures
In this research we use the Emotiv Epoc EEG headset to track emotions. The headset contains 14 electrodes spatially organized according to the International 10-20 system, moist with a saline solution. The detailed position of the measured regions is shown in Figure 4. The Emotiv system generates raw EEG data with 128 Hz sampling rate as well as the five frequency bands. The system uses internal algorithms to measure the following mental states: frustration, engagement, excitement, and valence. Although we don't have access to the system's proprietary algorithms, studies have provided evidence showing the reliability of its output [19]. We design our Neurofeedback system of Zoo Therapy based on these four emotional values.

RL-based Agent
In the past, our agent used the rule-based method. In this method, the agent will issue instructions to the animal based on the human-made feedback rules of the emotion that changes fastest per unit time among the four emotion values. However, this approach does not account the correlation between multiple emotions and cannot simultaneously show an increase or decrease in their value. The rules are artificially provided in advance, it is difficult to cover all changes in the emotional value of all users. Hence, we develop a reinforcement learning-based Neurofeedback system. The benefit of RL-based method is that it can learn the rules from the user's emotion in a flexible way, especially when the artificial rules are not accurate and hard to extract. Reinforcement learning emphasizes how to act based on the environment to maximize the expected benefits. It is inspired by behaviorism in psychology, that is, how an organism, stimulated by environmental rewards or punishments, gradually forms the expectation of the stimulus, and produces the habitual behavior that can obtain the maximum benefit. An agent must exploit existing experience to gain benefits, as well as explore unobserved status so that it can make better choices.
Our RL model has three components: agent, Zoo environment, and reward function. The agent is the decision-maker who will select an action from the action space given by Zoo Therapy and send it to the Zoo environment. The Zoo environment is the virtual animals and users which is the medium of agent interaction. The user's emotional value, as the agent's goal to maximize, is the reward. To fully reflect the user's emotions, we designed and trained a deep autoencoder model to encode 4 emotional values into one value to the latent representation of the current state of user emotions, called comprehensive emotional value. This value is used as an important component of the reward function. The reward function of this approach is defined such that if the user's comprehensive emotional value increases positively, then the reward is positive and viceversa. We adopted the open-source codebase of the Proximal Policy Optimization (PPO) algorithm [24] to train the model. Since the EEG changes of different users may change at any time and are unpredictable, our RL-based agent can perceive the user's emotion fluctuations, and when it finds that the user's comprehensive emotional value has decreased, it will immediately take corresponding actions to try to improve. Instead of giving instructions based on rules like the rule-based method at a fixed time.

Experiments
Our Zoo Therapy will be used to stimulate positive emotions and reduce negative emotions, thereby achieving the effect of slowing the progression of Alzheimer's Disease.
To analyze the effectiveness of our approach, we aimed to experiment the entire Zoo Therapy System with participants, unfortunately, due to the Covid-19 pandemic, experiments could not be performed with VR Zoo Therapy, but we tested the three research questions in this article separately. (1) Is it possible to interact by the user's voice in a virtual reality environment? We tested the Zoo Therapy in a non-VR setting with the speech recognition system on five real participants of different ages, genders, and accents. This test is mainly to verify whether the speech recognition system works in Zoo Therapy and how effective it is. We let users experience using 3D buttons or the speech instruction system to interact with animals, and then tell the researchers their overall experience and rating Zoo Therapy after use. (2) The Neurofeedback system can send corresponding instructions to the virtual reality environment in real-time to intervene on the user's emotions? We used the synthetic AD EEG data to train the agent and simulate the experiment of the RLbased Neurofeedback system. We generated 12 distributions of four different EEG emotions and the EEG data of AD patients comes from a similar project [20]. We used these 12 distributions to randomly generate a hundred sets of emotional values to simulate users and test our system. This set of experiments test whether the RLbased Neurofeedback system can learn rules from simulated user's EEG data and can give feedback when the emotional value changes. Simulated users can be roughly divided into three categories: like animals, don't like animals, and neutral. (3) Is it possible to reduce the user's negative emotions by interacting with virtual animals? We tested the Zoo Therapy environment in a follow-up project [23], it combined the VR Zoo Therapy with gesture recognition and rule-based neurofeedback system and proved the effectiveness by evaluating one participant's EEG emotion value. We started by equipping the participant with an EEG headset and then added the Fove VR headset in which we installed the Leap Motion devise (used for gesture prediction), and the participant started the immersive experiment.
In the whole process, the user's emotion value detected by the EEG helmet can reflect in real-time whether the user's negative emotions have decreased.

Results
The following results correspond to the experiments of each research question.
(1) Is it possible to interact by the user's voice in a virtual reality environment? We test Zoo Therapy with speech recognition system with five participants. All 5 participants said that the speech interaction method brought surprises to people. Although the virtual horse is in the computer, compared with a mouse click in 3D buttons on menu panels, the animal's direct response to the participant's voice is still delightful. The participants expressed that they prefer such animals and want to continue to interact with them. Participants rated the Zoo Therapy with speech command system from 0-10 subjectively in the "Evaluation" based on their own experience, the average evaluation of the five users was 7. Figure 5 shows the model's learning curve. In the test, the prediction accuracy of the speech recognition model for the user's voice instruction is 98%-99%.  Figure 7) and serve as an important part of the reward function of the reinforcement learning model. Autoencoding is first mapping the 4 emotions as a four-dimension vector into a one-dimension latent representation, and then reconstruct this fourdimension vector from the latent representation. It can be viewed as feature extraction and dimension reduction method, i.e., reduce the 4 emotions dimension to the one comprehensive emotional value, while still maintaining the reconstruction ability. The model is trained by minimizing the reconstruction error between the input four-dimension vector and the reconstructed four-dimension vector. Through 100 epochs of learning, it can be seen from Figure 6 that the model can quickly converge, and the MSE error of reconstructing 4 emotion values is 0.011. We used this model to train and test the RL-base agent. The behavior of RL agents can be divided into 3 parts.
For users who like animals, the RL-based agent gives proximity instructions, which account for about 63% of all instructions for this kind of user and 60-70% of the approach instructions occur when the user's emotion drops. For simulated users who don't like animals, the approach instructions given by the RL-based agent account for 59% of all instructions for this kind of user. The natural user command accounted for 54% of all instructions. The instruction to leave is issued more times to users who don't like animals. But the gap between the three types of people is not large.
: For the instruction of "stand up", the number of times sent to users who don't like animals is 120%-130% of the number of times sent to users who like animals. and show the system will bring animals closer to users who like animals, and less proactively approach users who do not like animals. Through the analysis, is the product of the combined effect of multiple emotions. For users who don't like animals, when the animal is very close to the user, the user's frustration value may rise. The other three emotional values may also rise. When the system chooses to stand up, the target location of this behavior is a certain distance away from the user. It will not be too close to the user to cause negative emotions but try to get closer to increase other emotional values. Through learning, the agent finds a strategy to balance the relationship between distance and multiple emotions to maximize the user's positive emotions. The agent has learned a potential feedback rule. Figure 8. 6 of 100 experimental results. The green line is the EEG curve of simulated users who like animals, cyan corresponds to users who don't like animals, and purple corresponds to natural users. RL-based agent: red markers. Rule-based agent: blue markers. The circles represent "come" or "eat", the squares represent "stand up", and the stars represent "walk" or "leave".
In addition, it can be seen from Figure 8 (blue markers) that the rule-based method sends instruction at a fixed time, and it is unable to give feedback on the emotional changes between the two instructions. Its characteristic is that it will give instructions in strict accordance with our rules. The RL-based method will give different strategies for different simulated users. From the red markers in Figure 8, the timing of the RL-based agent's instructions is more flexible. It doesn't need to wait for a fixed time but immediately gives feedback based on real-time emotional changes. It is more sensitive to emotional changes and can be changed at any time after the order is issued. The most important thing is that it can be generalized to a more comprehensive data set and learn new potential rules what we didn't know.
(3) Is it possible to reduce the user's negative emotions by interacting with virtual animals? According to the emotion value recorded in the cooperative project, we analyzed the mean frustration of the participant before, during, and after Zoo Therapy. Results show that, before the therapy, the mean frustration was 0.524, during Zoo Therapy, the mean frustration was 0.429, and after the mean frustration was 0.486. Figure 9 shows the difference between the mean frustration before, during, and after Zoo Therapy. Although this result is carried out under the gesture interaction method, in view of the positive feedback from users under the speech instruction system, we believe that the use of speech interaction and the enhanced intervention of the Neurofeedback system can also achieve the same positive effect of reducing the user's negative emotions.

Conclusion and future work
In this research, we aim to design and develop a virtual reality animal treatment that can replace real AAT for Alzheimer's disease, called Zoo Therapy, which stimulates the user's positive emotion feedback through the interaction with the virtual animal dog or virtual animal horse to reduce the user's negative emotions and relieve AD symptoms. The salient feature of Zoo Therapy is that in the 360-degree VR HD scenario, users can interact with the virtual animals through multi-interaction methods.
We created a VR environment in which users can interact with virtual animals. We designed three ways of interactions in the Zoo Therapy: 3D button, speech instruction, and the Neurofeedback system. Using the 3D button, the interaction is as simple as clicking on the required action for the animal. The speech recognition system using an automatic speech recognition (ASR) model that can accept trained voice instructions and the animals can follow them as if they can hear the users in real-life. The Neurofeedback system can instruct the animals to react as per the user's emotions.
Due to the Covid-19 epidemic, it was not possible to perform the experiments with real subjects. Hence, we performed preliminary experiments with some lab students and reported the results in this study. We get a good result for the one real AD patient. We received positive feedback regarding the design of the Zoo Therapy. The users were very excited with the horse's response and the overall experience of the Zoo Therapy elicited positive emotions in them. The Neurofeedback system was tested on a hundred simulated users, and it was able to instruct the animals by considering the user's current emotional state. All that show our Zoo Therapy will lead a positive emotion and might be used to reduce AD symptoms.
In the future, Zoo Therapy plans to invite patients to experience. In particular, the Neurofeedback system will use real patient EEG data for training and adjustment, to improve the accuracy of real-time intervention of the model in users' emotions. We will also try to extend it to relevant projects that can perceive users' emotions and give feedback, to investigate whether the model can autonomously learn new feedback rules and intervene with patients in different specific environments.