{'fraction_non_alphanumeric': 0.03343134490675474, 'fraction_numerical': 0.008136303218270432, 'mean_word_length': 4.63888258005905, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Probabilistic Human Motion Prediction via A Bayesian Neural Network

Human motion prediction is an important and challenging topic that has promising prospects in efficient and safe human-robot-interaction systems. Currently, the majority of the human motion prediction algorithms are based on deterministic models, which may lead to risky decisions for robots. To solve this problem, we propose a probabilistic model for human motion prediction in this paper. The key idea of our approach is to extend the conventional deterministic motion prediction neural network to a Bayesian one. On one hand, our model could generate several future motions when given an observed motion sequence. On the other hand, by calculating the Epistemic Uncertainty and the Heteroscedastic Aleatoric Uncertainty, our model could tell the robot if the observation has been seen before and also give the optimal result among all possible predictions. We extensively validate our approach on a large scale benchmark dataset Human3.6m. The experiments show that our approach performs better than deterministic methods. We further evaluate our approach in a Human-Robot-Interaction (HRI) scenario. The experimental results show that our approach makes the interaction more efficient and safer.


I. INTRODUCTION
Human motion prediction is an important and challenging topic in both computer vision and intelligent robotics communities which has been widely adopted to solve human intention understanding in human-robot-interaction scenarios. Supposing a robot is interacting with a human partner, for instance, handing a tool, playing games, or even taking care of the person, the interaction would be more efficient and safer if the robot could understand person's intention and accurately predicts person's future motion.
Currently, most of state-of-the-art human motion prediction algorithms [1], [2], [3] rely on Recurrent Neural Networks (RNN). Given an observed motion sequence, these methods only predict a deterministic result, which brings a number of serious problems in the Human-Robot-Interaction (HRI) scenario. First, the robot should not make only one prediction since human motions are diverse in real interaction scenarios. The similar observed motions may correspond to very different future motions due to different intentions. Second, the robot does not know how much should it trust the predicted results. Specifically, if the observed motion has never been seen before, the predicted motion is actually untrustworthy. Third, the robot does not know how long should it trust the predicted motion. As RNNs could generate arbitrary length of human motions and the predicted results become more inaccurate as time goes, there are only a period A probabilistic human motion prediction algorithm should not only predict many possible results but also need to choose the optimal one to help a robot make the most appropriate decision. of predicted motions trustworthy. Therefore, it is very risky for a robot to make decisions relying on a deterministic motion prediction algorithm.
A more reliable human motion prediction algorithm in the human-robot-interaction system should be a probabilistic model. Previous methods such as [4], [5] introduce Variational Auto-Encoders [6] or Generative Adversarial Networks [7] in the motion prediction algorithm which makes the model able to generate many possible results when given an observed motion. However, these methods could not further guide the robot to select the optimal prediction. In this situation, randomly select a predicted result is also risky. Thus, as illustrated in Fig.1, an ideal probabilistic model should not only predict many possible results but also need to tell the robot if it is risky to make predictions. Moreover it should give an optimal prediction to help the robot make the most appropriate decision.
For this purpose, in this paper, we propose a novel probabilistic human motion prediction model based on a Bayesian Neural Network (BNN) [8], [9]. The key idea of our approach is to extend the conventional deterministic motion prediction neural network to a Bayesian one. On one hand, as the BNN replaces the deterministic network's weight parameters with distributions over these parameters, our model could predict a number of possible future motions when given an observed human motion sequence. On the other hand, the BNN based motion prediction model could naturally model two kinds of uncertainties simultaneously, including Epistemic Uncertainty (EU) and Aleatoric Uncertainty (AU) [10]. The EU captures the ignorance knowledge about the prediction model. Therefore, it can tell the robot whether an observed motion has been seen before. For the unseen observations, the robot could refuse to make risky decisions. The AU could captures noise inherent in the predicted motion sequences, enabling AU to reflect the confidence of the predicted results. Therefore, it helps the robot to select the most trustworthy parts of predicted results.
In summary, our contributions in this paper are as follows: • We propose a probabilistic approach for human motion prediction in the HRI scenarios. Given an observed motion sequence, the proposed approach not only predicts a number of possible future motions, but also tells the robot if the observation has been seen before. It also determines the optimal result among all possible predictions to guide the robot to make correct decisions. • We evaluate our approach on a popular human motion prediction benchmark dataset Human3.6m [11] by designing a serials of experiments. The experimental results show that our approach performs better than deterministic motion prediction methods. • In a HRI scenario, we verify that the proposed approach makes the interaction more efficient and safer.

II. RELATED WORK A. Deterministic Human Motion prediction
In recent years, with the development of deep learning technologies, there are a number of approaches [1], [2], [3], [12], [13], [14], [15] which predict deterministic human motions by utilizing deep neural networks. For example, Encoder-Recurrent-Decoder (ERD) model [1] predicts the pose of human body based on the input videos. A Structural-RNN [3] combines the high-level spatio-temporal graphs. It can convert the structural transformation from spatiotemporal map into RNN network using a hand-crafted graph. Modified from typical RNN structure, Martinez et al. [2] propose a sequence learning model with residual connection. Their network directly predicts the velocity of human motion and integrates the velocity into the pose of the previous frame. Given an observed motion, these methods only predict a deterministic result, which is usually problematic in HRI scenarios.

B. Probabilistic Human Motion Prediction
Most previous probabilistic human motion prediction use non-deep learning approaches [16], [17], [18]. These methods usually rely on traditional machine learning algorithms such as Gaussian Process, Hidden Markov Model and Dynamic Forest Model. However, the crucial problem of these methods is that they cannot be applied to large scale datasets. Moreover, it is difficult for these methods to learn the variety of human motion patterns. Recently, with the development of Variational Auto-Encoder [6] and Generative Adversarial Networks [7], some probabilistic motion prediction approaches [19], [4], [5] which rely on deep neural networks, are proposed. For example, Butepage et al. utilize a conditional variational auto-encoder which makes the model able to generate many possible results by sampling variables in a latent space. Barsoum et al. [4] propose an approach which learns a probability density function of future human poses conditioned on previous poses. It also could predict multiple sequences of possible future human poses. However, these methods could not be directly adopted in HRI scenarios since they could not further guide the robot to select the optimal prediction. Under this circumstance, randomly select a predicted result is also risky. Different from previous methods, we propose a novel probabilistic human motion prediction model which based on BNN. Given an observed motion, our approach not only predicts a number of possible future motions, but also tells the robot if the observation has been seen before and also gives the optimal result among all possible predictions to guide the robot to make correct decisions.

C. Bayesian Deep Learning
Bayesian Neural Networks [20], [21], [22], [23] usually replace the deterministic network's weight parameters with distributions over these parameters, which gives network the ability to estimate the model uncertainty. Given an input, a BNN first samples weights from the weight distributions and then perform the forward calculation, which is always timeconsuming. Gal [24] finds that using Monte Carlo Dropout in a neural network during both training and testing could be approximately regarded as modeling Bernoulli distribution over weight parameters. Based on the Dropout BNN, Kendall and Gal [10] propose a method to capture two kinds of uncertainties, including EU and HAU. Kendall et al. [10] propose a framework combining HAU (input-dependent) together with EU. A framework without retraining to calculate the model uncertainty is proposed by [25], however, which cannot distinguish Aleatoric Uncertainty (AU) and EU.

III. METHOD
The goal of our approach is to make probabilistic and trustworthy human motion predictions for safe human-robotinteraction. In our approach, we represent human motion by a sequence of continuous 3D human skeleton poses. Each pose is represented as a vector that consists of the Cartesian coordinates of the joint locations. Figure 2 illustrates an overview of our model. It can be seen that our model consists of three modules, including a probabilistic future motion predictor, an unseen motion detector and an optimal motion selector. When given an observed motion sequence x −Tp:0 = {x −Tp , ..., x 0 } (abbreviated as x p ), our future motion predictor FŴ i with parameterŴ i first predicts a number of probability results (x i 1:T f ,σ i 1:T f ), i = 1, ..., N (a pair of mean and standard deviation, abbreviated as (x i f ,σ i f )), where x t = {x t,1 , ..., x t,J }, J is the number of human motion joint, and N is the number of samples, and the ground truth of future motion is x 1:T f (abbreviated as x f ), and T f is the maximum prediction length which is set to 2s. Then, the unseen detector calculates EU of the predicted results to determine if the observed motion has been seen before. For an unseen input motion, EU would be large, which prevents the robot from making risky decisions. Finally, the optimal motion selector calculates HAU for each predicted future motion. Consequently, the optimal prediction (x i * f ,σ i * f ) and its trustworthy parts (x i * 1:T i * ,σ i * 1:T i * ) could be identified, where i * is the index of the optimal prediction, and T i * is the optimal length of its trustworthy parts.

A. Future Motion Predictor
To generate probabilistic human future motions, we extend a previous deterministic motion prediction neural network Seq2Seq [2] to a Bayesian neural network which directly learns the distributions over the weight parameters q(W ). As a result, our future motion predictor can generate arbitrary number of possible results when given an observed motion sequence. Our model also consists of an encoder and a decoder. The encoder receives the input sequence x p and obtains a hidden variable h, and the decoder converts h into possible future motions. If the network after dropout samplingŴ ∼ q(W ) is FŴ i , our model can be described . The detailed network structure is shown in Fig. 3. Both encoder and decoder adopt Gated Recurrent Units (GRUs) [26] to recurrently make predictions over temporal domain. For the encoder, based on the survey results of [27], we first adopt a difference mechanism to capture the approximate velocity and acceleration of the current frame. Specifically, the acceleration can be obtained by the second-order difference of the previous three frames x t−2:t . Further we take the current position x t , velocity υ t = x t − x t−1 and acceleration a t = υ t − υ t−1 as the human dynamics of the current frame. Finally, the human dynamics [x t , υ t , a t ] is fed to the GRU cell to calculate the hidden state h.
For the decoder, the GRU predicts the speedυ t and the standard deviation of the positionσ t of current frame. The mean of the current position x is obtained by addingυ t and the input positionx t−1 . Then the human dynamics [x t , υ t , a t ] is fed to the GRU cell for predicting the speed and the standard deviation of next frame.
In our model, the Monte Carlo Dropout (MC-Dropout) is used for both training and testing. On one hand, [24] proves that Dropout is a reliable way to model the distributions over weight parameters. On the other hand, leveraging Dropout avoids directly sampling from weight parameter distributions, which is quite efficient for real time motion prediction systems. It should be noted that the difference between MC-Dropout and naive Dropout is that the mask of MC-Dropout on each RNN time step is the same. To simulate the unavailability of some key points caused by real sensors, our dropout unit takes the three coordinates of each key point as a whole.

B. Unseen Motion Detector
Knowing whether the observed motion has never been seen before can prevent the robot from making risky decisions. As our motion predictor is a BNN, we can further design an unseen motion detector which leverages the EU to identify the unseen observations. Specifically, given an observed motion x p , the unseen motion detector first calculate the EU D(x p ) by Eq. (1), where fŴ i is our recurrent neural network with sampled weights using MC-Dropout. fŴ i (x p ) is the prediction results, D(x p ) is obtained by subtracting the square of its mean (the right term) from its second-order moment of origin (the left term). In other words, D(x p ) is a function of x p , and D(x p ) obtained by estimating the variance of fŴ i (x p ). Then, the unseen motion could be easily identified by comparing the EU with a pre-defined threshold D thd . Obviously, the threshold is very important which could directly affect the robot's decision-making. To find a proper threshold. We provide a simple yet effective way. The EU of each x m in the dataset X = x 1 , x 2 ..., x M is D(x m ), m = 1, 2, ..., M . We sort D(x m ) to D (m) in ascending order, and the threshold is given by D thd = D ([0.95M ]) , where [·] is the rounding function. It means that 95% of the data in the dataset would be accepted by the detector. 0.95 is an empirical value that works well in most situations, because it can eliminate some noise data. So, if x p satisfying D(x p ) < D thd , it would be accepted, otherwise it would be rejected.

C. Optimal Motion Selector
For a motion sequence that has been seen before, the motion predictor generates N possible results. Then selecting the optimal prediction could help the robot make the most appropriate decision. For this purpose, we design the optimal motion selector which not only selects the best future motion x i * f from all possible predicted results, but also determine the maximal trustworthy length T i * that satisfies some security conditions.
As our future motion predictor is built on top of GRUs which recurrently make predictions, the HAU naturally reflects the noisy scale of each predicted frame. Therefore, we can not only find the optimal one from all possible motions, but also drop the corrupted frames for selecting the trustworthy parts from the optimal motion.
To capture the HAU for each frame of the predicted motion sequences, as shown in Fig.3, our model additionally output a standard deviation associated with the predicted human pose. The usual way is to modify the output dimension of the last layer of the network. Here we assume that the distribution of output is Gaussian N (x i f , (σ i f ) 2 ). The index i * of the optimal prediction (x i * f ,σ i * f ) of x p can be found by: Then, if there is a maximum T i * satisfying λσ i t < e max for t = 1, 2..., T i * , then T i * is the optimal length of the trustworthy parts. And we set the hyper-parameter λ = 1.28.

D. Overall Objective Function
Our model outputs the probability prediction at each time step. In the case of data independence, the objective function can be obtained by maximizing the posterior probability [28], [29]. We derive our loss function for BNN: The left term is weighted mean square error (MSE) between the predictions and ground truth in joint level, which is helpful for the variance to fit the prediction error from the data. The right term is a regularization term that helps to keep the variance at a reasonable value. The weight decay term is omitted here.

IV. EXPERIMENTS
In this section, we first evaluate the proposed approach on a large scale benchmark dataset Human3.6m [11]. We compare our approach with two baseline methods, including a state-of-the-art deterministic motion prediction model Seq2Seq [2] and a strong baseline Zero-velocity which takes the last input frame as all future predictions. Then we test our approach in a HRI scenario to demonstrate its effectiveness.
It worth noting that our Future Motion Predictor, Unseen Motion Detector and Optimal Motion Selector are denoted as FMP, UMD and OMS for short. We use "+" to indicate the combination of different parts. The numerical results of FMP and FMP+UMD are mean values since our approach could predict a number of possible results. The optimal numerical results are reported when we use the OMS.

A. Dataset
The Human3.6m is currently the largest available motion capture dataset which is widely used for evaluating motion prediction algorithms. It contains 7 actors performing 15 varied activities such as walking, eating, smoking, discussion etc. In this dataset, each human pose is represented by a high dimensional vector of 3D joint locations.

B. Implementation Details
All our experiments are conducted on Pytorch 1.4 [30]. The encoder of the FMP consist of a single GRU cell with 1440 neurons. The decoder consists of a GRU cell and a Fully Connected layer with 1440 and 51 neurons. We adopt the Adam optimizer [31] to train our model. The learning rate is set to 0.0005 and the weights decay rate is set to 0.0001. The batch size is set to 128. For each Dropout layer, the probability parameter is set to 50%. Our visualization results are rendered using matplotlib and open3d [32].

C. Unseen Motion Detector
We first conduct experiments to demonstrate how the unseen detector affects the predicted results on Human3.6m dataset. In this experiment, all methods are trained on action "walking", and then evaluated on three different actions, including "walking", "eating" and "sittingdown". We compare  our approach with the baseline methods using two metrics including the unseen samples Detection rate over and Mean Per Joint Position Error (MPJPE) on acceptance in terms of meters. The results are reported in Table I. The constant velocity [33] is not included in the table because of its serious deformation predictions on human motion prediction. First, we see that only with the help of UMD, the unseen test sample would be detected. There are 4.82% of the test samples in "walking" which have a large EU so that they are regarded as unseen samples. It is because the data distributions of the train set and the test set are not exactly the same. For these samples, our FMP has a lower confidence for making predictions. On "eating" and "sittingdown" test sets, 67.65% and 100% of the test samples are detected as unseen. The reason is that the action "eating" includes eating with sitting, eating with standing, eating with walking, which overlaps with the action "walking" partly. The action "sittingdown" includes sitting on a chair, sitting on the ground, etc. There is almost no overlap with "walking".
Second, on the "walking" test set, our FMP achieves better MPJPE results compared with the baselines. It shows that training the motion predictor with the constrain of uncertainty could also improve its performance. on the "eating" and "sittingdown" test sets, both Seq2Seq and FMP are worse than Zerovel., which indicates that making prediction for unseen motion sequences is even worse than predicting the future motion as a static pose.
Third, it can be seen that combining the FMP with UMD achieves the lowest MPJPE results on "walking" and "eating". It refuses to make predictions for "sittingdown" since all test samples are regarded as unseen. In other words, using the UMD could effectively identify the unseen motion sequences to prevent FMP from making bad predictions.

D. Optimal Motion Selector
We further conduct experiments to intuitively demonstrate the effectiveness of the Optimal Motion Selector in this section.
Given an observed motion sequence, our FMP predicts a number of possible future motions, in which each frame is attached with a predictedσ. By simply calculate the mean σ for each motion, the OMS finds the optimal one. Fig.  4 shows the visualization results of the predicted motions which correspond to a same observed motion. It can be seen that the optimal result, which has the smallest σ = 0.078, is most close to the ground-truth motion. Fig. 5 shows our OMS also could accurately predict the maximal trustworthy length according to the HAUs. In this experiment, we set the error threshold to 0.20m. In other words, we think the predicted frames whose MPJPE is  larger than 0.20m need to be abandoned. By comparing the predicted results with the ground-truth motions, we get the real maximal trustworthy length which equals to 1360ms. By using the predicted HAUs, OMS predicts the maximal trustworthy length as 1240ms, which is very close to the real maximal trustworthy length. It worth noting that the real maximal trustworthy length could hardly be achieved in real time HRI scenarios since the ground-truth motions are usually unavailable. In this situation, OMS provides a reliable solution.
We present the numerical results to show the effectiveness of OMS in the next section.

E. Future Motion Prediction
In this experiment, we compare our approach with the baseline methods on all 15 actions in the Human3.6m dataset. The MPJPE results are reported in Table II. First, we see that FMP performs better than the two baseline method since it achieves a lower MPJPE at each time step. This is consistent with our previous conclusion that training the motion predictor with the constrain of uncertainty could improve the predictor's performance. Second, with the help of UMD, there are 4.30% of the observed motion sequences are regarded as with a high EU which are directly dropped without making any prediction. Therefore, the MPJPE on acceptance of each time step is further decreased. Third, when we continue to introduce the OMS, our approach achieves the best results, which indicates the OMS could select the optimal future motion among all possible motions. Here, the OMS only finds the optimal predicted motion without dropping any frames with high HAU.

F. Safe Robot Trajectory Optimization
In this experiment, we test our approach in a HRI scenario. Specifically, we demonstrate whether our approach could help the robot avoid collisions when interacting with a human. Inspired by CHOMP [34], we define our trajectory optimization objective function by replacing the original distances with the predicted uncertainty. Fig. 6 visualizes the trajectories of the robot and human in a HRI scenario. In the left figure, we see that the robot collided with the human, since the robot did not predict the human's motion. The middle figure shows the predicted human motions after the robot observes the human for 2s. Then, as shown in the right figure, by taking the future human motion into consideration, the robot optimizes its trajectory for avoiding collision.

V. CONCLUSION
In this paper, we propose a probabilistic approach for human motion prediction in human-robot-interaction scenarios. The proposed is based on a Bayesian neural network which aims to learn the distributions over weight parameters. On one hand, it is capable of predicting a number of possible future motions when given an observed human motion sequence. On the other hand, it could capture two kinds of uncertainties simultaneously to tell the robot whether the predicted results are trustworthy. In this way, it helps the robot to make the most appropriate decision. We first validate our approach on a benchmark dataset and then evaluate it in a HRI scenario. The experimental results demonstrate the effectiveness of our approach.