{'fraction_non_alphanumeric': 0.030093851209191273, 'fraction_numerical': 0.009999763598969291, 'mean_word_length': 4.916363636363636, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 1, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Robust and Subject-Independent Driving Manoeuvre Anticipation through Domain-Adversarial Recurrent Neural Networks

Through deep learning and computer vision techniques, driving manoeuvres can be predicted accurately a few seconds in advance. Even though adapting a learned model to new drivers and different vehicles is key for robust driver-assistance systems, this problem has received little attention so far. This work proposes to tackle this challenge through domain adaptation, a technique closely related to transfer learning. A proof of concept for the application of a Domain-Adversarial Recurrent Neural Network (DA-RNN) to multi-modal time series driving data is presented, in which domain-invariant features are learned by maximizing the loss of an auxiliary domain classifier. Our implementation is evaluated using a leave-one-driver-out approach on individual drivers from the Brain4Cars dataset, as well as using a new dataset acquired through driving simulations, yielding an average increase in performance of 30% and 114% respectively compared to no adaptation. We also show the importance of fine-tuning sections of the network to optimise the extraction of domain-independent features. The results demonstrate the applicability of the approach to driver-assistance systems as well as training and simulation environments.


Introduction
With 1.3 million deaths and 30 million injuries occurring yearly worldwide, road traffic accidents are the main cause of death for people aged 15-29, and represent a cost to governments of, on average, 3% of national GDPs [1]. A high proportion of those accidents occur during manoeuvres such as changing lanes and turning [2]. Advanced Driver Assistance Systems (ADAS) aim at increasing road safety by taking partial control of the car or by providing the driver with extra information when such manoeuvres could be dangerous [3]. It has been demonstrated that, thanks to recent developments in deep learning and computer vision, it is possible to predict manoeuvres a few seconds in advance and with high accuracy, by monitoring the driver's behaviour inside the vehicle and using information from the car itself (e.g. speed) and the environment (lanes configuration, presence of intersections, etc.). In particular, advances in Convolutional Neural Networks (CNN) now allow accurate extraction of head-, face-, and gaze-related features from videos [4,5], while Recurrent Neural Networks (RNN) enable the models to take into account the temporality of an action, i.e. the order in which certain actions are performed or specific events occur [6].
However, while most recent proof-of-concept models for manoeuvre anticipation have achieved good results, little attention has been given to the problems arising from the practical implementation of such systems. One of the main concerns is their ability to generalise on subjects that were not part of the original training set. In real world applications, it is likely that ADAS installed on commercial cars will be "blind" to the driving style of new subjects. Re-training such systems may not feasible due to the lack of labelled examples for a new driver. While there are examples of deep neural networks trained to anticipate actions and objects in videos through unsupervised learning [7], most of the examples found in literature are not able to learn temporal relationships between features, or work exclusively with video inputs [8]. This is a problem in manoeuvre anticipation tasks, since it has been demonstrated that multi-modal inputs and the temporality of events are crucial to obtain quick and accurate predictions. Moreover, if they are not re-trained, classic deep neural networks tend to not generalize well when features in the test and training sets have different marginal distributions. Examples could include cases in which the driver has peculiar driving habits or mobility limitations, so that, for instance, they cannot turn their head fully. It is however plausible to assume that there exist common patterns and latent features in the actions of most drivers which are 1.1 Aim

INTRODUCTION
common regardless of the vehicle, driving style, or situation [9].
This assumption represents the basis of domain adaptation, a type of transfer learning technique which enables previously trained models to adapt to other datasets containing unlabelled observations [10]. Amongst the various approaches, domain adaptation applied to deep neural networks has been proven to be extremely effective to learn domain-invariant features from the input data even when the labels of the target distribution are unknown [11]. This enticing result is obtained by optimising a discriminative classifier while simultaneously maximising the loss of an auxiliary domain classifier. This technique has been proven to work not only with image inputs, but also with time-series observations [12]. Using this method, we hypothesize that a model can be trained to find features in sequential input data which are not only discriminative of specific manoeuvres, but also shared between the training set (large and labelled) and a fully or mostly unlabelled small test set -such as video segments of a new driver performing unknown manoeuvres. Finding such latent features may, however, be difficult when the inputs are multi-modal time-series. This type of input is common in state-of-the-art manoeuvre anticipation models, which integrate the driver's behaviour with information from the vehicle and the environment -such as lane configuration, car speed, and GPS data [13]. The network does not only need to learn the dependencies within a single time-series, but also across input sequences with possibly very different resolution (sparse vs. dense) or even data types (e.g. categorical vs. continuous). An effective sensory fusion approach is therefore required.

Aim
Given the problem of adaptation and generalization capabilities of multimodal models for driver-assistance systems, we present an investigation of the application of the domain-adversarial training method to implement domain adaptation on a Recurrent Neural Network for manoeuvre anticipation. More generally, this paper proposes domain adaptation as a promising approach to improve the performance and generalization ability of machine-learning-driven ADAS. We aim at showing that domain-adversarial models are particularly beneficial in situations where either the drivers or the driving settings -or bothmay differ considerably to those used to train the model.

Method and Structure
In order to evaluate our approach, we designed three experiments, which are introduced in the following paragraphs and discussed more in depth in Section 6. Before presenting the results of the experimental work, we provide a bibliographical review of the most recent published work on action prediction, maneuver anticipation, and domain adaptation. We then present a technical overview on Recurrent Neural Networks. Finally, we describe the architecture of our models and the most salient features of both the Brain4Cars and our own dataset, including a detailed explanation of the data collection process. The experimental set-ups consist of the following: Experiment 1 We first propose an expansion of the architecture proposed by Jain et al. [13] (Brain4Cars), which includes the driver's gaze as an additional input, features a higher number of stacked recurrent layers, and performs enhanced sensory fusion by combining Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU) layers, amongst other improvements. We replicate the experiments presented in the original paper, training and testing the LSTM-GRU model on the Brain4Cars dataset using simple 5-fold cross-validation. This experiment is aimed at showing that the performance of our architecture is comparable to state-of-the-art models for maneuver anticipation in non-adaptive tasks. Additionally, we also test the model on a new set of driving videos obtained using an immersive virtual simulation setup, providing a performance baseline upon which to evaluate the results of the subsequent experiments.

Experiment 2
We then present a Domain-Adversarial RNN, inspired by Ganin et al. [11], in which our LSTM-GRU network from Experiment 1 serves as the feature extractor. We train and test the DA-RNN using a cross-validated, leave-one-driver-out approach on individual drivers from the Brain4Cars dataset, comparing its performance to the non-domain-adaptive LSTM-GRU trained without the target driver. adaptation in commercial ADAS, driving training set-ups, and simulation environments.
The crucial contributions of this work can thus be summarised as follows: • An improved LSTM-GRU Neural Network architecture for manoeuvre anticipation.
• An original dataset of observations from a driving simulation setup.
• A proof of concept for the application of a Domain-Adversarial RNN for domain adaptation on driving data, which employs the LSTM-GRU architecture to classify observations from the new dataset.

Related Work
The majority of studies relying on driving data do not concern themselves with the adaptation of the system to different datasets or their ability to perform well on new drivers; those who aim at anticipating manoeuvres are not an exception. Samples from all test subjects are often shuffled together before the training-test split [13]: while this enhances a classifier's performance, it will likely cause it not to generalise and scale well in real-world applications. Once an ADAS is installed on a car, it will have been trained on a large number of drivers, but there is no guarantee that it will work well with a completely new subject without re-training. In this work we tackle this problem by applying the domain-adversarial training technique to encourage a Recurrent Neural Network to adapt to a smaller, unlabelled sets of driving data in which the features may have different marginal distributions. To obtain meaningful results we also decided to design an improved model for manoeuvre anticipation, addressing some of the shortcomings of previous work. In this section we provide a brief survey on the most recent research on action prediction and manoeuvre anticipation, as well as on the latest techniques in the area of domain adaptation, with special focus on those applied to deep learning and sequential data. For an in-depth review of the general field of domain adaptation, we suggest the papers by Jiang [14] and Patel et al. [15]

Action Prediction
Predicting future actions differs from simple classification tasks, in that the aim is to anticipate an event with as little data as possible. One way to do this practically is by setting a threshold, processing the data at each time-step of an input sequence of observations, and only making a prediction when the probability of the corresponding output is above that threshold. To do so, a network needs to be able to remember past states and observations within a single time-series, in order to learn the temporal relations between latent features in that sequence. Most recent examples of action anticipation models do this by using LSTM-or GRU-based architectures [16,17,18,19]. LSTMs and GRUs are specific types of RNN units which are able to remember and forget previous states through a number of modulating gates [20,21]. They have been proven to solve the problem of vanishing gradients, which affects heavily "vanilla" recurrent networks, and thus are able to perform very well with long time series [22]. This aspect makes them more suitable to process sequential observations than regressive models such as Gaussian Mixtures and non-recurrent Neural Networks, or Hidden Markov Models (HMMs), which assume that each observation's probability only depends on the current state and are thus not suitable for modelling contextual effects and long sequences [8]. Despite these shortcomings, HMMs have been shown to produce promising results, especially compared to approaches that do not employ deep learning [13]; they could therefore provide an excellent alternative in cases where the high computational requirements of RNNs cannot be met, for instance in mobile applications. A further improvement on the HMM approach is represented by Markov Decision Processes, which have been shown to provide reliable and accurate predictions for long-term driving risk inference [23].
In order to make predictions within a few time-steps, rather than simply classifying an action when the whole sequence has been processed, a recent trend has been to implement custom loss functions which exponentially increase with time. Later classification mistakes are penalised more heavily than earlier ones: the model is thus encouraged to provide a confident prediction as soon as possible. In multi-class tasks such as manoeuvre anticipation, these functions are often modifications to the standard cross-entropy loss. Aliakbarian et al. [24] included this type of time-dependent loss in a multi-stage LSTM architecture that manages to predict actions accurately with only a small percentage of video sequences, by learning context-and action-related features independently. Chan et al. [25] used the exponential loss only for positive examples in a binary classification task to predict driving accidents, also including it in a LSTM-based RNN.

Manoeuvre Anticipation
Models for manoeuvre anticipation are similar to those for action prediction; the differences are the specificity of the subject's movements (mostly head and eyes) and the type of contextual information obtained from the environment.
At the time of writing, the most recent and possibly complete approach to anticipate driving manoeuvres has been proposed by Jain et al. [13] (Brain4Cars).
Their model implements an LSTM-based architecture with high-level sensory fusion to process multi-modal observations, using facial landmarks, head pose, car speed, GPS information and lane configuration. An exponential loss function was also included, which was proven not only to successfully encourage early predictions, but also to act as a regulariser. In addition, they provided a complete dataset of driving videos, showing the driver and the outside environment in the few seconds before turns and lane changes. Because of Brain4Cars' promising results, in this work we used their architecture and dataset respectively as benchmark and for evaluation purposes.
An area of improvement identified by the Brain4Cars team lies in the addition of eye tracking information to the model's inputs. Gaze direction, in fact, has been shown to generally correlate with the direction of the subsequent movement [26,27,28,29]. Including gaze direction as an additional feature vector is particularly important in the context of manoeuvre anticipation: eye movements to look in the mirrors or at objects on the road may not necessarily be accompanied by a movement of the head, but may provide information about the direction of a consequent manoeuvre. It has been argued that the choice not to implement it is justified by the difficulty of obtaining accurate measurements of gaze direction without using specialised hardware [13]. Fletcher and Zelinksy [30], for instance, successfully implemented an ADAS which analyzes driver inattentiveness using gaze tracking, and Ravichandar et al. [8] used prior probabilities based on the eye gaze to enhance the accuracy of their actionprediction model; both works were carried out using ad-hoc tracking cameras.
However, recent studies have shown that it is in fact possible to estimate gaze direction accurately from videos captured by regular high definition cameras [31,32].
While it is clear that eye tracking may provide additional benefits to infer the intentions of a driver, adding additional sensors to the model requires optimal sensory fusion in order for deep networks to perform well [33]. Indeed, a share of the recent literature has focused on improving the integration of the multimodal data coming from both inside and outside of the car. For instance, Doshi et al. [34] used a relevant vector machine (RVM) model to detect lane-change intent; they exploit the ability of the RVM to obtain a sparse data representation of the dataset, thus performing sensory fusion by automatically choosing discriminating features from multi-modal signals. Tawari et al. [35] developed a "Merge and Lane Change Assist" system in which sensory fusion is achieved by encoding all constraints (spatial, temporal, as well as legal) into a compact probabilistic representation [36]. Jain et al. [13] used a solution similar to those proposed by Sung et al. [37] as well as Yang and Eisenstein [38], which is not to join the features before feeding them to the network, but rather to concatenate their high-level representations through learnable layers of the neural network. This is a very simple and scalable solution that only requires simple modifications to the architecture of the network.

Domain Adaptation
Domain adaptation is a type of transfer learning where two domains (source and target) share their feature space but have different marginal distributions [12]. It attempts to solve the issue of enabling a model trained on a certain dataset (source domain) to perform well on a differently distributed dataset, of which the labels are completely or partially unknown [9]. This problem has been investigated in computer vision [39,40,41] and natural language processing [42] using a variety of different approaches to reduce the discrepancy between the two domains, including alignments of the subspaces [41], parameter augmentation [43], domain-invariant projection [44], and instance re-weighting [45]. Purushotham et al. [12] point out that these techniques are not able to capture the temporal dependencies in sequential data, and those which do (for instance through a Bayesian approach [46] or RNNs [47]), cannot accurately infer non-linear relationship. Deep learning approaches, on the other hand, have proven successful in capturing time dependencies and complex, non-linear, domain-invariant relationships through domain adaptation. Examples include marginalised denoising autoencoders [48], ad-hoc CNN architectures [49], and feature embeddings [50].
Out of the deep learning approaches, one of the most elegant and easy-toimplement solutions is the Domain-Adversarial Neural Network (DANN), developed by Ganin et al. [11]. DANNs perform domain adaptation by learning domain-invariant features through a neural network architecture composed of three sections: a feature extractor, a discriminative classifier, and an adversarial domain classifier, whose loss is maximised through a special gradient reversal layer. The domain classifier's role is to encourage the feature extractor to find latent representations of the features which are domain-invariant [12]. In particular, this type of domain adaptation aims at creating a common subspace for the source and target domains, so that the trained model can classify examples from the target domain without having access to the labels of the target's training set [11]. The ability to adapt in an unsupervised manner is particularly appealing for manoeuvre anticipation models, as it would allow them to generalise well to new drivers simply by retraining them using a set of unlabelled driving videos as the target domain. While this method has been originally carried out using a CNN for the feature extraction stage, Purushotam et al.
applied the technique of adversarial training on a RNN (R-DANN) and a variational RNN (VRADA), in order to capture temporal relationships from one domain to the other [12]. Our paper introduces the same concept applied to manoeuvre anticipation, expanding on the simpler R-DANN architecture and using an LSTM-GRU network as the feature extractor.
Lastly, since finding domain-invariant features in multi-modal time series can be challenging and computationally expensive, it has been suggested that finetuning the feature extractor section on the source dataset can yield to an improved performance without sacrificing domain invariance [51,11]. This has been shown to be effective even in non-adaptive cases of time-dependent features [24], and its application to the manoeuvre anticipation problem will also be demonstrated in this paper.

Model for Manoeuvre Anticipation
In order to apply the domain-adversarial approach to manoeuvre anticipation, we first propose an improved RNN architecture based on LSTMs and another closely-related type of gated layer, the GRU. Both prevent the gradient of the loss function from either vanishing or exploding during backpropagation thanks to the activation functions of the layers' gates, which are learnable and create sums of activations over which the derivatives can distribute [13,52].
The gradient can thus propagate for a long time, allowing long time-series to be processed. Before illustrating our model, we provide a brief explanation of these two layers.

LSTM and GRU
The structure of a typical LSTM can be seen in Fig.1a. It consists of a memory cell (c), which allows information to be accumulated over long sequences,  . The process is defined by the following equations: where W * are the weights and b * the biases. is the Hadamart product, also known as element-wise or point-wise vector product. [53,13].
The GRU, pictured in Fig.1b, is similar to the LSTM, but is lacking the memory

Anticipation Framework 3 MODEL FOR MANOEUVRE ANTICIPATION
cell and the output gate [54]. It is defined by the following equations: As it can be seen, the number of operations computed at each time-step is lower than for the LSTM. It was also found that GRUs outperform LSTMs in specific situations, for instance when no dropout is used [55,53]. This can be explained by a lower tendency to overfit due to the reduced complexity. While the introduction of dropout makes LSTM better choices in most situations, using GRUs lowers computation and training time by reducing the number of learnable parameters. They thus represent a valuable option.
Throughout this paper, LSTM and GRU operations will be referred to as:

Anticipation Framework
To build our model we followed the framework for manoeuvre anticipation proposed and defined by Jain et al. [13], which we summarise here. The inputs to the model at training time are represented by N time series in the form x t is received by the model at every time-step. A probability threshold p th is chosen, so that when, and only when, any y j ≥ p th , a prediction of the respective manoeuvre will be made. At t = T , if no manoeuvre has been predicted with sufficient confidence, the default action (going straight) will be predicted. and data processing pipelines are described more in detail in Section 5. The structure of our architecture, shown in Fig.2, was inspired by the sensory fusion approach proposed in previous work, in which the inputs are not concatenated before going through the network, but rather after the first recurrent layer(s) as high-level features [13,37,38]. We enhanced this idea by incorporating the concept of action-and context-dependent features proposed by Aliakbarian et al. [24]. They showed that, for a similar type of anticipation problem, this structure performs better than two parallel recurrent layers which disregard the action-context distinction. The concatenation occurs at two different places in the network: first the representations of the head features φ t are concatenated with those of gaze γ t , after each of them has gone through an LSTM layer (Eq.12 and 13). Their concatenation is then passed through a GRU layer (Eq.14). The rationale for this choice is as follows: when an event can be classified in a main type (manoeuvre vs. going straight, i.e. no manoeuvre) and a sub-type (each of the four manoeuvres), a two-layered RNN architecture enables each of the two layers to specialise in the two predictions. This idea was proposed and successfully tested by Xiao et al. [56]. In our case, the first LSTM learns the main event type, while the following GRU learns which manoeuvre is performed. A GRU layer was preferred to a second LSTM in order to limit the increase in complexity. The output of the GRU layer is then concatenated with the hidden output of a third LSTM layer whose inputs are the environmental features η t (Eq.15). This concatenation is then fed to a fully-connected dense layer (Eq.16) [13]. Finally, the last softmax layer provides the classification probabilities for the five classes (Eq.17).

Network Architecture
Because of the high number of learnable parameters (> 18 × 10 4 ), dropout was applied to both the recurrent and dense layers of the network. A dropout of 0.6 was set to the LSTM and GRU recurrent connections (recurrent dropout), while a dropout of 0.7 was set to the output of every layer [57,58]. Additionally, the bias of the recurrent layers was initialised to 1, in order to improve the model's performance and training [53,22].
In order to encourage the model to anticipate early, we implemented the loss function shown in Eq.18, which weighs each term of the cross-entropy categorical loss with an exponential term, as a function of time. The loss will be greater at larger values of t, thus rewarding the network for predicting the right class as early as possible [24,25,13]. y j t represents the probability of event j computed by the model at time-step t.
The network was trained through backpropagation with gradient-based optimization; we chose the Adam optimiser [59] for its simplicity, robustness, and low computational cost [60]. The learning rate was set to 1 × 10 −3 ; the other parameters were set to the values suggested in the original paper: β 1 = 0.9, β 2 = 0.999, = 1 × 10 −8 , without temporal decay. We implemented additional regularization by using early stopping, with a patience of 80 epochs. Our DA-RNN, pictured in Fig.3, is made up of three sections, divided in two main branches. The first section employs the LSTM-GRU network described in Section 3.3 (except for the last softmax layer) as a feature extractor. Its role is to learn the latent relationships between the features in the observation sequences. After the latent features are extracted, their hidden representation is fed into two branches: the first one is a discriminative classifier for manoeuvre anticipation, composed of a single softmax layer; the second one is an adversarial domain classifier. The key part of the latter is the gradient reversal layer: during the forward pass, the input is left unchanged, while during backpropagation, the gradient is negated and multiplied by a constant, λ (lower section of Fig.3). The loss of the domain classifier is thus maximised, thereby encouraging the feature extractor to find representations of the features which are domain-invariant [12]. Conversely, the label classifier's loss is minimised in order for the features to also be discriminative of the manoeuvres. The manoeuvre loss L y is the time-dependent exponential function in Eq.18, while the domain loss L d is the regular binomial cross-entropy loss function. The total loss is given by Eq.19; the effect of λ is discussed more in detail in Section 6.

Domain-Adversarial RNN
To train the model, half of each input batch is filled with samples from the source domain, and half from the target domain [11]. Since the latter, by assumption,

Brain4Cars Dataset
The dataset used as a benchmark for testing our anticipation architecture was developed by Brain4Cars [13]. refer the reader to the original work by Jain et al. [13]. The dataset is publicly available on the Brain4Cars website. 1

Our Dataset
In order to build our own dataset for a more extensive evaluation of our model and the domain adaptation approach, we created a driving simulator setup following a standard structure for research [61,62]

Feature Extraction and Processing 5 DATA COLLECTION AND PROCESSING
samples for a domain adaptation problem; in a real-life situation, target sets will always be considerably smaller than source sets.
Our dataset has been made available as Open Access on Zenodo with the following DOI: http://dx.doi.org/10.5281/zenodo.1009540.

Feature Extraction and Processing
To extract features from the videos of both datasets we used OpenFace, an open-source toolkit that provides facial landmark tracking, head pose estimation, and gaze tracking, from videos and images, using a combination of Conditional Local Neural Fields (CLNF) and CNNs [32]. It is capable of high performance real-time tracking using regular cameras, making it an attractive option for ADAS. Once the data was extracted from the videos, we followed the feature processing pipeline defined by Jain et al. [13], with slight modifications to binning intervals, labelling of the environmental features, and feature scaling.
The head pose was also included using the Euler angles representation [R x (α pitch ), R y (α yaw ), R z (α roll )] as it was shown to improve the performance [13,67]. Overall, the histogram features and head pose form the head features φ ∈ R 13 .

Eye Gaze
Information about the driver's gaze was also obtained through OpenFace's output, which provides a 3D direction vector for each eye, normalised and in world coordinates. We took the average components of both eyes, and applied a Butterworth low-pass filter (4th order, sampling frequency of 30 Hz, cutoff {−1 < δx ≤ −0.5; −0.5 < δx ≤ 0; 0 < δx ≤ 0.5; 0.5 < δx ≤ 1} pixels, in the image space. Identical bins were chosen for δy. In this case we used the direction components rather than the inter-frame velocity since we found that it correlates better with the driver's intention. We define γ ∈ R 8 as the gaze features.

Context-related Features (Environment)
The environmental context is expressed by η ∈ R 4 . The first two features are boolean variables expressing the presence of a lane to the left and to the right of the car, respectively. The third feature is also boolean, and indicates the presence of an intersection ahead and in the near proximity of the car. For the purpose of this study, these three values were labelled manually for each observation. In a practical implementation, this information could be extracted automatically through lane detection algorithms [68,69] fused with GPS data.
Finally, the fourth value is the speed of the car in km/h. This value is provided in the Brain4Cars dataset, and was measured by the physics engine of the game in our dataset.

Experiments
In this section we present the results of the experiments conducted on the Brain4Cars' and our dataset, using the features extracted according to the method described in Section 5. We first tested the network for manoeuvre anticipation on the two datasets separately, shuffling samples from all drivers before the training-test split, as done by Jain et al. [13]. We show that the new architecture and additional features yield to an improvement in the performance of the model. We then investigated the applications of domain adaptation to our model via domain-adversarial training, in two ways: a) employing a leaveone-out approach on each driver in the Brain4Cars dataset, in order to study the possibility of personalising a model in a partly-unsupervised manner; b) using the entirety of the Brain4Cars dataset as the source domain and our dataset as the target domain, in order to study how the approach works with different feature distributions. In both cases, we attempted to increase the performance of the models through fine-tuning -meaning initializing the weights of certain layers with weights from a pre-trained model-, and we studied the effect of varying the value of the domain loss multiplier λ.
To evaluate the results, four measures were used. Three of them provide the multi-class classification score for the predicted action, namely: precision, recall and F1 score [70]. The fourth measure is the time-to-prediction (TTP), representing how many seconds before the onset of the manoeuvre the prediction is made. When calculating precision, recall, and F1 score, the "going straight" predictions are not considered, as it is considered the baseline state. The probability threshold was set to 0.9 for all experiments, since this value was shown to yield the most confident and quick predictions [13]. This means that at each time-step t, a prediction is made only if at least one of the 5 outputs of the models is ≥ 0.9. We trained all models with batches of size 128.

Experiment 1: Manoeuvre Anticipation
The first experiment was performed using the LSTM-GRU architecture described in Section 3.3 on the Brain4Cars dataset, without domain adaptation, in order to evaluate its performance as a feature extractor. Samples from all drivers were shuffled, and a test set was set aside taking 15% of the total observations. 5-fold cross validation was used, with the 5 validation subsets each making up a different 20% of the training set. Training and validation sets were normalised jointly before the split. Following Jain et al.'s approach, we augmented the training set by extracting subsequences of random length (T sub ) from the original observation, with 50 < T sub < 150. These additional samples were used as additional training examples, thereby adding redundancy and additionally decreasing the risk of over-fitting [13]. More sub-samples were taken from under-represented classes, thus balancing the class ratios. For the Brain4Cars dataset, this led to a total of 2160 training samples; 312 for ours.
The results are shown in Table 1. We compare our performance with Jain et al.'s LSTM model [13]. It can be seen that our network performs better for all manoeuvres, reaching a higher F1 score as well as a higher time-to-prediction. We theorise that this improvement is due partly to the additional information given by the gaze direction. Indeed, excluding the gaze-related features, the performance of the model was observed to drop by up to 1-1.5%. However, we attribute the improvement especially to the enhanced sensory fusion (including the action-context structure). Indeed, a parallel structure in which the hidden representations of all three inputs are concatenated after the initial recurrent layers performs marginally worse than Brain4Cars' model. This shows that simply adding more features does not necessarily improve the performance of a model if it is not accompanied by rational modifications in the architecture.
In addition, it was noticed that the added complexity of the network caused a strong tendency to overfit, despite the high dropout, the early-stopping criteria, and the redundancy in the training data. However, it is likely that the number of samples is not large enough; bigger datasets should alleviate this problem and further enhance the model's performance. Lastly, we report that including the car's speed as part of the environmental features was detrimental to the performance of the model, and, for this reason, it was excluded in the subsequent experiments. This may be due to the fact that, especially in sequences leading to lane changes or driving straight, maneuvers may not be correlated with specific ranges of speed values; further analysis should be carried out on the subject. However, we observed that excluding any of the other features used by Jain et al. [13] from either dataset worsens the performance considerably.
This was confirmed experimentally through 5-fold cross-validation.  The same model was also tested on our dataset, using a similar procedure.
The results, illustrated in Table 2, show that the performance is lower than for the Brain4Cars dataset. This is most likely due to the lower number of samples,  Overall, these results show that our model represents an improvement over the most recent LSTM-based approach in manoeuvre anticipation, and prove that our dataset can be used reliably with a comparable performance. Moreover, it was shown that fine-tuning can be a powerful approach to enhance the model's capability if a larger, more reliable dataset is available.

Experiment 2: Domain Adaptation on Different Drivers
In order to investigate how a trained model can learn to adapt to a small set of unlabelled driving videos belonging to a subject not included in the training set, we manually separated the videos of each individual driver in the Brain4Cars dataset from each other. We then ran our DA-RNN using observations from all drivers except one as the source domain, while, as the target domain, we used the samples from the remaining driver. Once trained, the model was tested on a set of samples from the target domain not included in the training set, all from the same driver. This was done for each driver separately, and the results were averaged. In addition, we fine-tuned the networks by initializing the weights of the LSTM-GRU feature extractor by pre-training them on the source domain.
The results are shown in Table 3  weighting of the losses from the two classifiers yielded the best performance.

Experiment 3: Domain Adaptation on Our Dataset
The third experiment was performed to attempt a transfer of information between the Brain4Cars data and our dataset. Being able to adapt a model to observations in which the features are distributed very differently can be important in applications such as driver training, Virtual Reality simulations, and autonomous driving research. Fig.5 shows that the two distributions are indeed very different: the head movements made by the drivers in a real-world driving scenario have a higher within-manoeuvre variance than those driving a simulator, but a smaller variance between the different manoeuvres. The fact that the subjects were asked to perform the same movements as in a real-setting, however, makes it plausible that it is possible to find latent features which are shared by both datasets. Moreover, the fact that fine-tuning the model using one dataset improves the performance on the other provides additional support to this theory.  The first striking observation is the extremely poor performance of the model without adaptation. We infer that when the marginal distribution of the features in two domains are very different, a non-adaptive model fails to generalise.
The DA-RNN, in comparison, performs better, with an increase in F1 score of 12 percentage points and an increase in time-to-prediction of 0.2 s. The most critical improvement was registered when the feature extractor was fine-tuned on the source domain, with an F1 score of 62.7%. The overall performance is still lower than the case of Experiment 2, when the drivers belonged to the same dataset (i.e. driving in similar conditions), but much higher than the noadaptation case. These relative improvements in performance are comparable to the ones found in recent works performing similar domain adaptation tasks, such as the ones reported in the original DANN paper by Ganin et al. [11].
Overall, these results confirm the hypothesis that there exist latent features shared by datasets of observations in which similar driving tasks are performed, but in largely different settings; moreover, they highlight the need of an adaptive approach for the practical implementations and personalizations of ADAS.
The models were created, trained and tested using Python 3, using the deep learning framework Keras 2.0 with the TensorFlow 1.2 backend. Network training was run on a machine with CPU Intel Core i7-7700K and GPU NVidia GTX 1080, and 32GB of memory.
The code of the architecture of the DA-RNN can be found at https:// github.com/michetonu/DA-RNN_manoeuver_anticipation.

Conclusions
In this paper we proposed and tested a Domain-Adversarial Recurrent Neural Network for adaptive driving manoeuvre anticipation. Trained on a large source dataset of driving observations, our DA-RNN is able to adapt to smaller, The second evaluation consisted in using the Brain4Cars dataset as the source domain and our dataset as the target domain, with the same feature space but a different feature distribution. We reported an improvement in F1 score by 114% (32.7 percentage points) compared to the non-adaptive case, when the model's weights are fine-tuned. Overall, we demonstrate that the domain-adversarial approach represents a promising approach to increase the flexibility and generalization capabilities of commercial ADAS through domain adaptation.
We conclude that non-adaptive models are not able to generalise well in contexts a) where the target driver was not part of the training set, and b) where the features in the target domain have a very different marginal distribution, which is the case when the driving set-up in the target set differs from the one used to collect the training data. Adaptive models will therefore be necessary for ADAS installed in commercial vehicles, and will prove helpful in virtual simulations and driving training tasks. In order to further validate the approach in the context of assisted driving, additional tests should be conducted on a larger target dataset obtained in real-life conditions. Larger and more diverse driving video datasets will enable even higher performances, as one of the bottlenecks of our approach was found in the network's tendency to overfit, due to the complexity of the architecture and the relatively small size of the datasets. This work sets the bases for further research aimed at enabling adaptive deep neural networks to reach performances comparable to fully supervised models.

Acknowledgments, Contributions and Funding
We would like to thank the members of the PERCRO Lab who volunteered for the data collection process, namely L. Peppoloni, G. Dabisias, L. Landolfi, and F. Brizzi. A huge thank you also to J. von Kügelgen for the technical support on domain adaptation and thorough proofreading of the manuscript.
Finally, we would like to extend our appreciation towards the creator of all the software used in this work, as well as to the authors of the Brain4Cars dataset for making it available online for free. We strongly believe in open-source software and open access to data; we therefore decided to also publish and share our dataset for free use.