{'fraction_non_alphanumeric': 0.03167089805929687, 'fraction_numerical': 0.006971040061964801, 'mean_word_length': 4.7100737100737105, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Deep Generative Modelling of Human Reach-and-Place Action

The motion of picking up and placing an object in 3D space is full of subtle detail. Typically these motions are formed from the same constraints, optimizing for swiftness, energy efficiency, as well as physiological limits. Yet, even for identical goals, the motion realized is always subject to natural variation. To capture these aspects computationally, we suggest a deep generative model for human reach-and-place action, conditioned on a start and end position.We have captured a dataset of 600 such human 3D actions, to sample the 2x3-D space of 3D source and targets. While temporal variation is often modeled with complex learning machinery like recurrent neural networks or networks with memory or attention, we here demonstrate a much simpler approach that is convolutional in time and makes use of(periodic) temporal encoding. Provided a latent code and conditioned on start and end position, the model generates a complete 3D character motion in linear time as a sequence of convolutions. Our evaluation includes several ablations, analysis of generative diversity and applications.


Introduction
Individuals perform complex motions with the motivation of certain goals to reach or meet, energy efficiency, swiftness, as well as physiological constraints. Yet, even for identical tasks, the resulting motion is subject to natural variation. Modelling all kinetic, physiological and neural relations is an elusive goal. We suggest a method to capture all these aspects computationally, based on machine learning.
We use a deep generative model, that, provided with a start and and end goal of a motion, produces the in-between animation. Instead of giving one answer -or even worse, an average of all possible answers-it captures natural variation by enabling sampling, i.e. producing an infinite stream of possible motions samples, from the conditional distribution of all motions that fulfill the goals. This model is trained adversarially [11], supervised on raw scanned 3D motion trajectories and the start and end goals.
We contribute a dataset of 600 human goal-driven 3D motions, to sample the 2 × 3-D space of 3D source and targets. While temporal variation is often modeled with complex learning machinery like recurrent neural networks or networks with memory or attention, we here demonstrate a much simpler approach, convolutional in time and making use of (periodic) temporal encoding [27]. The structure of this work is as follows: after reviewing related literature in Sec. 2, we introduce our approach Sec. 3, involving data capture, and a convolutional GAN using time coordinate encodings. Results are presented in Sec. 4 before concluding in Sec. 5.

Previous Work
There are several broad strategies that have been employed in order to solve the problem of generating novel motions from a limited set of motion captured examples. The first distinction is between kinematic and physics-based approaches. In kinematic approaches, plausible pose sequences/motion are generated without attention paid to physical properties. In physics-based approaches, a physical model is implemented and physics-based control signals (e.g. joint torques) are selected to create physically-plausible motion that responds to elements of the environment. Our approach belongs to the former category, so we start by reviewing works of this kind in this section.

Non-deep learning approaches
Within the category of kinematic approaches, a variety of techniques have been employed to generate motion examples. Older approaches involve stringing together and blending motion clips, like in [23], which uses a motion graph [17,3,18] constructed from mocap samples to generate new motion by interpolating two paths through the graph. These two paths with a satisfactory interpolated motion are found through A* search.
Another way of blending motions is using kernel-based methods (e.g. [22,19]) that learn an embedding of highdimensional full-body motion in a low-dimensional latent space and can generate new, intermediate/blended motions by selecting points in the latent space.  Figure 1. We capture human subjects performing a reach-and-place action over time (horizontal) for several configurations of goals (rows), denoted pink and cyan. A video of the action is shown left. Off-the-shelf 3D capture produces samples of a distribution of motions, conditioned on goals (middle). We propose a neural network to generate samples from this goal-conditioned distribution of motions (right). Data distribution samples are subject to spatial noise, not present in generated samples. Samples and goals shown are random.
A recent approach [25] uses shallow, 3-layer neural networks to learn multiple experts that output the next frame of motion given the previous frame (a motion prediction network), and learns an appropriate weighting of the experts based on the phase and goal of the motion (a gating network). These networks are trained on an augmented motion capture dataset.
In recent years, however, deep learning techniques have shown promise in many applications,not only for motion denoising and motion reconstruction [16], but also for motion generation. We review some recent advances in the next section.

Deep learning generative approaches
Recurrent neural networks (RNNs) are a type of deep neural network that specialize in generating sequential or time series data. Long short-term memory units (LSTMs) are an alteration that overcomes some of the vanilla RNN's shortcomings, like the vanishing gradient problem. These networks have shown promise at generating human motion sequences [9,10]. RNNs or LSTMs can be combined with a generative adversarial network (GAN) structure, as [8] do to generate gestures.
While RNNs specialize in sequential data, there is some evidence that temporal convolution may be more effective than RNNs, at least for prediction involving long-term memory [5]. Similar to our work, [15] use temporal convolution, or convolutional neural networks, to learn from mocap examples and use it to generate motion sequences which they apply to mocap cleanup. Their design is an autoencoder, typically introducing smoothing, which we prevent by making use of adversarial training. As an aside, for phased motions, such as walking, another interesting approach is to rather than learn temporal information, is to instead provide the phase of the motion as input to the network, as shown by [13] for motion frame prediction.
Another possibility is to use normalizing flows, a technique similar to GANs that allows direct maximum likelihood estimation during training. Henter and colleagues [12] combine normalizing flows with LSTMs in order to generate human and animal locomotion. This technique has also been applied to generating conversational gestures [2].

Extensions
Techniques for learning patterns from motion examples can be combined with other techniques in order to generate more complicated sequences of motions. For example, Ling and colleagues [20] use Variational Autoencoders to learn a motion space, but combine it with reinforcement learning to generate motions to accomplish higher-level tasks. [14] use the convolutional neural network from their 2015 work and use a feed-forward neural network to connect it with highlevel control parameters like motion trajectories to give the user more control over the generated motion.

Style transfer from video/images
Another promising avenue for generating novel motions is to use motion from non-mocap sources in order to drive or alter an existing motion capture clip or image. Deep learning techniques have proven useful here as well. [1] uses temporal convolution to learn motion information and combines it with time-invariant convolution to learn style information, which allows the means and variances of mocap motions like walking and jumping to be modulated by a style code derived from an unrelated video file. Instead of transferring video style to a motion capture motion, [24] use traditional convolutional techniques for processing images to transfer motion from a video to an input image, allowing a novel video to be created. Output of the generator is a motion (top). This motion is fed into a discriminator D, shown right. Learning maximizes confusion of that discriminator with respect to real and generated data i.e. motions of the data distribution (bottom).

Physics-based approaches
Physics-based approaches fall into two categories: approaches that incorporate motion capture as a reference for natural-looking motions, and pure physics-based approaches that do not make use of motion capture data. In the first category are works by [6] and [21], both deep reinforcement learning (DRL) approaches that incorporate closeness to motion capture as part of the reward function. The second category includes work by [7] on generating dressing motions using physics simulation and deep reinforcement learning as well as the approach by [29] that generates locomotion using a DRL method that penalizes asymmetry in the loss function.

Our Approach
We first explain how we represent the action goals (Sec. 3.1) and the resulting human reach-and-place action (Sec. 3.2), how to acquire the data (Sec. 3.3) before introducing our generative model (Sec. 3.4).

Input representation
We represent the motion constraints (i.e. goals) as a sequence of 3D vectors. As we are investigating reach-andplace motion only, this space is parameterized by a pair of 3D-vectors g ∈ R 3×2 = G. The first vector qualifies where the object is to be picked up, the second where it is to be placed.

Output representation
We represent each reach-and-place action as a tuple a = {s, m} ∈ A of 3D shape s ∈ S and 4D motion m ∈ M where A, S and M are the spaces of action, shape and motion to be defined below.
Our aim is to design these spaces to be simple enough to define convolutions on them and ultimately apply a convolutional GAN.
Shape We assume Kinect's standard topology. The human shape s = {d i } ∈ S = R nn is represented by n n = 25 nodes, a vector of distances in 3D world-space. In other words, this is a stick figure, with each stick representing a defined distance between a pair of nodes, whilst also having the flexibility to change orientation arbitrarily.
Motion We will make use of two interchangeable representations of motion: absolute positional motion, and relative orientational motion, both to be explained next.
First, absolute positional motion is a sequence of world space positions m a = {p i,j } ∈ R 3×nn×n k where every node i has a position for each of the n k animation frames j. This is the form in which we will acquire data, how samples of the data distribution look like and hence what the discriminator of a GAN will need to work with.
Second, relative motion is represented as time-varying direction m r = {ω i,j } ∈ R 3×nn×n k , a sequence of direction vectors. The orientation is relative to a coordinate frame defined from each node and its parent. We first compute a matrix to align the z axis with the direction from every node's positions at the first frame to its parent's position, also in the first frame. The generator-the most relevant step of our approach-will work with this representation. Note, that directions vectors are unit; multiplying them with the length of a bone will provide the position of a node.
The root node at time step 0 is considered to be at the origin in world-space. Beyond time step 0, the root node is further represented as a time differential: the change of relative position per time step.
The procedure described above can be used to convert from absolute to relative motion and vice versa, denoted aToR(m a , s) and rToA(m r , s). Note that both steps are differentiable, without learned parameters.

Data Acquisition
We directly capture our data distribution by acquiring 3D trajectories from a Kinect, recording at 30 Hz. Within each sequence we record 25 marker, or node, positions representing the major joints of the human body. In total, 600 sequences with durations of around 100 frames were captured. We also acquire the goal positions.
All actions were performed by one male person, age 24, with no reported physiological conditions and on the same object, a box of 200 g. In this work, we do not generalize across subjects, kinds of goal objects, or any domain other than the domain of interest (goal start/end position).
All motion is recorded in world coordinates, forming the data distribution, denoted Y. Samples from that distribution are noisy; a Kinect commonly makes mistakes in detecting pose. For example, limbs may be recorded in entirely wrong orientations, and parts of the body jitter and jump. While our results are not free from artifacts, we found those sampling artifacts to vanish, as in a denoising auto-encoder [28] but without the smoothing, thanks to our adversarial design.
We compute the shape s, i.e. the node distances, from the Kinect scans directly as the average of the distances between a node and its parent across all takes and all frames. Further, whitening is applied to the data by removing the intra-action mean and dividing by intra-action variance.

Learning
We perform adversarial [11], i.e. unsupervised, learning of a generator network to map from a latent coordinate and a pair of goals to a complete action, such that a discriminator network cannot tell apart samples produced by the generator and samples drawn from the data distribution.
More formally: where E x∼X [·] is the expected value operator sampling distribution X with sample x, D is the discriminator to classify an action as belonging to the data distribution or not, G is the generator, mapping the latent code z from the latent space Z to a complete action. Both generator and discriminator share the tunable parameters θ. Rather than use the GAN loss as originally conceived, we instead make use of the Wasserstein loss [4], which remedies the issue of vanishing gradients and allows for much more stable training.
Discriminator Recall that the discriminator has to accept samples y from both the data distribution Y as well as from the generator distribution G(z). As we prefer to generate in relative space, we transform the data samples before showing them to the discriminator. The input to the discriminator D(y|g) ∈ A → R is an entire action y and a goal g condition. Output is a single We only look at the absolute space-time-tensor m a for motion. Hence, when using this discriminator, shape s is just a time-constant set of values, informed by gradients, that together with a motion, can explain the data. In theory, the generator might produce implausible shapes that then with implausible motion generate the data. This however does not happen. networks tend to find the simplest explanation (parsimony) which very often is also the truth: network parameters making up "funny" skeletons and even more "funny" motion might be harder to find, perform worse, or simply do not even exist, than the ones that actually cater to the inductive bias. Using relative motion, hierarchical convolutions and quaternions, might be for human action, what the inductive bias of Deep Image Prior [26] was for images.
This task is performed using a convolutional encoding of m r and g. We start with a (n n + (2 × 3)) × n k tensor that stacks the time-varying orientation onto the two 3D goal vectors repeated over all keyframes. At each level of this encoding, the input is first convolved with a bank of filters, second passing every filter result through a non-linearity. The coefficients of the filter and the non-linearities (leaky ReLU) are the learned parameters θ. For image processing, these convolutions are two-dimensional, e.g. 3 × 3. For a skeletal animation we use 1D convolutions in time instead, i.e. 3 × 1. Note that convolution is translation-invariant, i.e. the same input produces the same output, regardless of where it is in time. This has benefits, but also drawbacks as explained next.
Additional to the action itself, we make use of coordinate encodings. Such an encoding adds additional features to the input of a convolutional neural network. While classic convolution encodes what a feature is, coordinate encodings hint to where -as we deal with time this means whena feature is present. In practice, this means a seemingly trivial change: at the first level, convolution will not only get the 3D positions, but also their time code.
A direct coordinate encoding has the drawback that it is very hard for a NN to capture high frequency response to that coordinate, first noted with respect to attention in language processing [27]. Hence it has been suggested, to also include periodic functions of coordinates, i.e. not only t, but also sin(t), sin(2t) etc. in the feature vector. We do so for two octaves. The operator coordCode computes these periodic signals and a linear ramp and adds the three channels to the tensor.
Such a detector is strong as it can find evidence for being fake at all time positions (convolutionality), whilst it is also informed about absolute positions as well (positional encoding) and finally consolidates information across all scales (hierarchical).
Generator The generator maps a latent coordinate z and the goal g to a relative action representation m r . We mirror the design of the discriminator, a conditioned convolutional encoder, to a conditioned convolutional encoder-decoder.
Directly producing 3D-vectors can unfortunately produce direction vectors, that are not unit vectors. We don't want to ask for the normalization from the network, neither do we want tolerate deviations from the desired stick-figure length. Hence, we simply normalize every direction vector before it is being used, a differentiable operation. Note, that it is not sufficient to work with a network that generates only two degrees of freedom, e.g. x and y and then solve for the third one, z, as there are two unit vectors x, y, z and x, y, −z.
The generator also uses coordinate encodings both on the encoding and on the decoding step and on all levels.
For input we use a latent space of size 200, which during training is drawn from a normal distribution with µ of 1 and σ of 100. We then simply concatenate the start and end goal coordinates (scaled between -1 and 1) to create an input vector of size 206. The first layer is a fully connected layer which projects the input vector of size 206, to a tensor of size 600 × 4. The subsequent upsampling and convolutional layers give an output equivalent in size to the discriminator input, (analogous to m r ) 75×32 time steps.
Tbl. 2 gives the detailed architecture of the network.

Results
We report qualitative results as well as comparison to a range of ablations.
Qualitative results Qualitative results are seen in Fig. 3. We see that the start and end pose are faithful human poses and the in between ones are plausible. They reach both goals in a reasonable posture and blend between them for all body parts. The weakest aspect are the feet: while upper extremities move plausible, the feet can sometimes appear to slide on the ground. Note, how the results are not perfect, but non-noisy, while samples from the data distribution have noise, e.g. for some frames, it can happen that a leg or arm is flipped, indicating the system acted as a denoising generative model. Fig. 4 shows the distribution of results when the goals are held fixed and different samples are drawn by changing the latent code. Despite the goals being constant, the animation however is different, while each is still a plausible motion reaching the goals, but in different ways.
Ablations Fig. 5 studies three ablations focusing on the main contribution that allows using convolutions from motion.
The first ablation (Fig. 5, top) is to not use coordinate encodings. We find that the skeleton has rigid bones, but the motion itself is subject to low and mid-frequency noise. The most salient mistake is to not make distinct gestures like footsteps, but everything occurs in a temporally-smooth underwater motion. This probably is, as the NN has to make up functions from boundary effects that support temporal locality. Using coordinate encodings, this effect is drastically reduced, as again best seen in the supplemental video.
The second ablation is to handle the root node like any other node: relative in space, but not relative in time (Fig. 5,  middle). This results in a patchwork of gestures that are plausible per-se but do not fit together.
The third ablation (Fig. 5, bottom) is to use absolute 3D positions throughout. This makes the implementation much simpler and for our exposition one may simply assume aTor and rToa to be identity. We find, that samples of this representation lack rigidity: the distance along bones is free to change. The NN does a reasonable job in maintaining a rigid ensemble stick-figure, but that comes at the expense of adhering to a constraint that is easy to enforce by-design.
From these ablations, which are best also seen repeated in the supplemental video, we conclude the representation, Figure 3. Results of our method (right) for five different goal configurations (rows) shown (left). The first action picks an object from the ground and the character stretches far up to place it. After the place-action, a slightly unnatural, round, pose for the legs is noticeable. Note how the character steps back to bend down. In the second action, the character leans forward to place the object at head level. In the third action, only one hand is used to hold the object. The fourth action has a wide horizontal extend, resulting in a turning of the shoulder to reach left-down and place right-up. In the final action, this is compared with a back-to-front motion, involving foot-stepping and some rich details like a horizontally tilted hip. Note: the goal placement and motion is best seen from the supplemental video. discriminator and generator are good examples of learning with hard-coded physical constraints.

Conclusion
We have presented a generative model of reach-and-place motion. Starting from human motion labeled by goals, the model, provided new goals, produces new motion.
We have reported qualitative results, demonstrating several aspects of our approach are superior to important ablations. Regrettably, as with many generative models, quantifying the success is difficult: we cannot simply use an L2 between any reference and what we generate, while existing measures of diversity, such as FID score are relative to image-specific properties. Accordingly, we have shown samples from the generated and data distribution -to be judged on their own, not in respect to a reference-in conjunction with ablations.
Present circumstances dictate that it is quite difficult to capture large amounts of motion capture data, however post pandemic it will be possible to capture data from more subjects and motions to generalize the model further. In future work, we would like to extend to different forms of goals, different subjects and objects to move, as well as use the model in other applications such as motion denoising, or motion in-painting.