{'fraction_non_alphanumeric': 0.025688073394495414, 'fraction_numerical': 0.00764525993883792, 'mean_word_length': 4.786198761427308, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Vehicle localization combining non-linear state observer with artificial neural network

The developing of autonomous drive is needed to make people life more comfortable and safer, and one of the important skills to make possible the reliability of the all control system is a good localization of the vehicle. In this study, a no-linear state observer was developed using the Unscented Kalman Filter (UKF) algorithm, to estimate the global position, global orientation, and local speeds of a car inside a known path. A characterization of the sensors input measures was made and the measures of longitudinal and lateral vehicle speed were added using an Artificial Neural Network (ANN) trained in simulated manoeuvres. In this way, it was possible to reduce the error that the observer make on the estimation of the lateral vehicle speed, and so of the side slip angle, making possible an improvement of the control activity. To assess this increase in performance, a Montecarlo analysis was made comparing the architecture proposed, ANN+UKF, with state observed, UKF, with no input measure of lateral speed. The tests were done in co-simulation environment of Vi-Grade’s CarRealTime software and Matlab-Simulink.


Introduction
The need to reduce situations that endanger people's health and safeguard their wellbeing encourage the automotive sector to develop control system that prevent human faults. But to allow these control systems to improve their performance a proper estimation system is needed to know as accurate as possible the actual states of the vehicle. For these reasons, in this study a comparison of a no-linear state observer with and without a measure of the lateral speed of the car was made trying to reduce the side slip angle estimation errors in limit of adherence manoeuvres.
The side slip angle estimation involved many research projects because of its important in the vehicle dynamics stability controls developed in the last years [1] [2]; as well as its importance in the development of autonomous driving systems [3]. The problem is that a direct measurement of the car's speeds implies the used of expensive sensors not usable in commercial car. So, in the state of the art, it is possible to find different methodologies to overcome this problem and implement a system that doesn't involve the use of direct measurement [4] [5].
The most commonly strategies consist of: a model based approach with state observers; and a black box approach using the artificial intelligence capability of approximate any functions. The first strategy cited has been widely investigated by researchers. Some authors implemented a closed-loop state feedback observer [6], others a non-linear observer based on Lypunov asymptotic stabilization function  [7], and others a sliding mode observers [8] [9]. However, with all these methods, the estimation is strongly influenced by the uncertainties of the vehicle dynamics. To reduce the errors made, adaptive methods are developed [10], [11], even if the most common method to estimate side slip angle was the Kalman Filter (KF). It was used in many different ways: linear KF [12], and non-linear such as Extended Kalman Filter (EKF) [13] [14] [15] and Unscented Kalman Filter (UKF) [16] [17] [18]. However, the UKF is preferred over the linear KF because it reduces modelling error by representing non-linear dynamics, and over the EKF because it is only reliable for systems which are almost linear within the operating frequency range, and become unstable due to the need to calculate the Jacobians at every time step. Anyway, modelling errors cannot be completely eliminated with model-based observers.
A completely different approach is the one based on machine learning techniques and, specifically, supervised learning. The main algorithm used in the literature is Artificial Neural Networks (ANN) which have been demonstrated to be capable of approximating any function [19]. For this methodology, too, different approaches were employed with regard to both the architecture of the layers and the input quantities. Some authors preferred to use only the inherit platform sensor measures, vehicle accelerations and yaw rate [20]; and others that increase the number of input considering also the steering wheel angle and the wheels speed improving the network capability to extrapolate the vehicle model [21]. If these examples implement a deep feed-forward neural network, better results are obtained with Recurrent Neural Network (RNN) layer, being able to learn the temporal links of the various signals [22].
However, even this strategy have issues because it needs to have a big data train to cover all the car's working range. Therefore, a mixed approach [23] can ensure to have the advantages of both the strategies: training the ANN to predict the car speeds on a set of manoeuvres with different combinations of lateral and longitudinal vehicle dynamics, and using the network output as input measures of the UKF, that can consider the vehicle dynamic and correct the lack in generalization of the network.
To evaluate the beneficial that the ANN bring to the UKF developed to an autonomous car as side slip angle estimator and car locator, an in-depth statistical analysis was done through the Montecarlo analysis.
After the introduction done in the first Section, in the second Section the state observer, UKF, developed will be explained with the dynamic model used and the simulated sensors characterization; in the third Section the architecture, the training phase and the results of the ANN used will be shown; in the end Section the Montecarlo analysis will be explained, and the achieved results and the conclusions shown.

State observer
The state observer developed is composed of an Unscented Kalman Filter (UKF) [24] able to estimate the actual global position and orientation, and local longitudinal and lateral speed of a car inside a known path. The UKF is a non-linear state observer that uses the unscented transform to propagate the so-called 'sigma points', a group of symmetrically distributed points around the previous estimated system states which contain the information of the expected mean value and variance of the system. Representing the non-linear system in discrete time form with noise as follow: where x k ∈ R n represents the state vector, u k ∈ R m is the input vector, y k ∈ R q the measurement vector, and the process noise v k and measurement noise w k are white Gaussian noises, i.e. zero mean and R v k , R w k as process and measurement covariance matrices, the UKF algorithm can be presented considering the state vector x k with mean valuex k and covariance Q k , and the following steps: (i) initialise mean value and variance matrix: wherex 0 is the a posteriori estimation of the expected mean value for k = 0 and Q 0 is the a posteriori estimation of the variance matrix for k = 0. (ii) Calculate sigma points χ k−1 : where λ is a scaling parameter, the constant α determines the spread of the sigma points around x k−1 and is usually set to a small positive value. The constant k is a secondary scaling parameter.
To calculate the square root of covariance matrix Q k−1 , the Cholesky factorization was used for which a Hermitian positive-definite matrix B can be decomposed as B = LL + with L being a lower triangular matrix with real and positive diagonal terms. (iii) Time update by transforming the sigma points with the non-linear functions: and computing the a priori estimation of the expected mean valuexk, variance matrix Qk and measurement estimationỹk: where the W i weights are: where γ is used to incorporate prior knowledge of the distribution of the state vector (for Gaussian distributions,γ = 2 is optimal). (iv) Measurement update by computing the measurement estimation variance Q ykyk and covariance matrix Q xkyk betweenxk andỹk: and finally, calculating the Kalman gain K k , a posteriori estimation of expected mean valuex k and variance matrix Q k :

Vehicle model
The model used by the state observer is a double-track vehicle model [25] shown in Figure 1. The state vector used is: Where X,Y are global car position on the track; θ is orientation of the car, i.e. yaw angle; vx, vy are local vehicle longitudinal and lateral speed; r is yaw rate; and α ij are lateral slips of the four wheels, where ij stands for front-left, front-right, rear-left, rear-right. The evolution of these states from the previous step to the actual one is described by the following time-discrete equations: Where F x/yf l , F x/yf r , F x/yrl , F x/yrr are the longitudinal and lateral wheel forces, a, b the front and rear semi-wheelbase, t the car track, rel the lateral wheels relaxation length, δ f the front wheel steering angle, and dt the sample time choosing to be 0.001 s. Thanks to this model, the state observer is able to reduce the error made in the estimation of the unknown variables combining the measured signals and their noise dispersions with the model signals and their dispersions. The dispersion of the model and of the measure is found by a tuning process in co-simulations between CarRealTime dynamic vehicle model and Matlab-Simulink implementation of the state observer.
The measurement vector used by the state observer is: Where X,Y are global position of the car inside the path measured by the GPS sensor; θ is global orientation of the car inside the path, measured by the yaw rate integral, reset to the camera heading angle measurement and global orientation of the road lines every time that the car is straight and the side slip angle is approximately zero; r, ax, ay are vehicle yaw rate, longitudinal and lateral acceleration, measured by the IMU sensor; vx, vy are longitudinal and lateral local vehicle speed, measured by two strategies that are compared in this paper, showed in Figure 2.
The first one doesn't have the lateral speed as measure signal but as a state of the filter, and measures the longitudinal speed by the longitudinal acceleration integral, reset on the mean wheels speed at a specfic time interval during acceleration or uniform drive, as shown in the equation below: The second strategy involved the used of an Artificial Neural Network (ANN) able to estimate the actual value of the longitudinal and lateral speed knowing the car acceleration ax, ay, the yaw rate, r, the steering wheel angle, δ, and the wheel speeds, ω ij . The ANN developed will be explain in the next chapter.
To better represent the real behaviour of the input and measure signals, a characterization of their noise was made by experimental tests, Figure 3.

Performance of the State Observer
The UKF developed without the combination with the ANN was tested in a set of manoeuvres, performed in co-simulation environment of Matlab-Simulink and vehicle dynamic software of VI-Grade, CarRealTime.
The results obtained in the estimation of the side slip angle are shown in the Figure 4 and underlines a good performance about the mean absolute value of the side slip angle estimation, 0.29 deg, but shows the limit of the filter in the prediction of the vehicle dynamic at the limit of handling manoeuvres, where the max absolute error achieves 1.5 deg, that considering a maximum value of side slip angle admissible for safety of 6 deg represents approximately 25% of the value.

Artificial Neural Network to car speeds estimation
As shown in the previous Section, by the tests made, a not acceptable error on the lateral speed estimation has been seen in the filter working at limit of adhesion situations. For this reason, it was thought to develop and use as signal measure inputs an ANN [22], that trained in a varied set of car dynamics conditions can estimate the longitudinal and lateral vehicle speed reproducing the non-linearities that a simplify Where ax, ay and r are the signal measured by the inertial platform, δ is the steering angle at the front wheels, and ω ij are the four wheels speed. The structure of the NN is shown on Figure 5, where: • Sequence layer: is the Input-Layer and is responsible for feeding inputs into the network in the form of a sequence; • GRU layer: is the first Hidden-Layer and is a Recurrent Neural Network (RNN) able to link signals in time by remembering past instants; • Fully Connected layer: is the second Hidden-Layer and can speed up the training phase and make the network recognise similar sequences by compacting the data; Then, the ANN, structured as described, was trained using a set of maneuvers representative of vehicle dynamics, on low and high friction condition, carried out using software simulations and simulations at a static driving simulator. The Data-Set used to train (70% of the data), validate (15% of the data) and test (15% of the data) the ANN is listed in the Table 1 and the ANN parameters are shown in Table 2.

Performance of the Artificial Neural Network
To asses the performance of the ANN, not only the manoeuvres defined by Table 1 were tested but than the ANN was implemented in Matlab-Simulink and was tested in co-simulation with the dynamic vehicle software, CarRealTime, to evaluated its real-time performance.  The results obtained in side slip angle estimation by testing the ANN in a Step Steer manoeuvre with high lateral load, and in a Sine Sweep manoeuvre, taken from the Data-set, are shown in Figure 6, where acceptable errors are achieved in longitudinal and lateral speeds estimation, summarized in Table 3. In fact, the mean and maximum absolute errors achieved are significantly reduced compared to the UKF by up to 10% error over a maximum of 6 degrees side slip angle proving that the network has been trained in a good way and had learn the input and output link. Testing the network with on-line run of the vehicle model, the results shown that the network generalization worsen as in the step manoeuvre of Figure 7 where the maximum error made is double the previous one.
This proved that the neural network probably has generalisation problems and that the training dataset needed to be extended to represent all dynamic driving conditions of the vehicle, thus lengthening the development time of the estimator.

Validation of the mixed approach
In this Section, firstly the simulation environment and then the testing methodology used to validate the robustness and accuracy of the mix-approach algorithm will be shown. The benefits of the proposed algorithm for both the longitudinal velocity and side slip angle estimation will be described.
In the results shown, the two strategies compared will be named as: • UKF: localization algorithm composed by only the state observer without the ANN; • UKF-ANN: localization algorithm composed by the mix between the state observer and the ANN.

Simulation environments
The tests were made in lap-top Intel(R), Core(TM) i7-7700HQ, CPU @2.80GHz and Ram 16 GB choosing a sample time of 0.001 s. The ANN and the UKF are implemented in Matlab-Simulink and to consider the vehicle dynamics, co-simulation was run between Matlab-Simulink and the Vi-Grade CarRealTime software that ensures the real-time dynamic response of the complex 14 DoF vehicle model.

Testing methodology
Since the main problem of the neural networks is to guarantee that the ANN provides an acceptable estimation error also outside the testing manoeuvres, and the problem of the UKF is to be able to describe the dynamic uncertainties linked to the tyres, it was thought to join the two systems to compensate the respective lacks of precision. In order to validate and verify the potential of this architecture under varying initial conditions and vehicle dynamics, a statistical analysis was carried out using the Montecarlo method [26]. This analysis is a statistical one and involves running the estimation system many times for the same manoeuvre by initializing the noise of the input variables in a random and always different way. During the state observer development phase, it was concluded that side slip angle estimation was most affected by variations in longitudinal speed, lateral acceleration and grip conditions. Therefore, it was decided to compare the robustness and accuracy of the UKF-ANN architecture with the UKF by performing a series of step manoeuvres where vehicle speed, steering angle and grip were varied. The manoeuvres repeated 50 times for the Montacarlo analysis are shown in Table 4 and a description of the results is shown in the next subsection.

Estimation results
For each type of manoeuvre repeated by randomly initialising the noise present in the input signals, the Root-Mean-Squared-Error (RMSE) performed in the 50 tests was calculated, thus ensuring the reliability of the estimate. The RMSE obtained from the two configurations was then compared as the influence parameters varied. About longitudinal speed estimation, as shown in Figure 8, the use of the neural network as input to the UKF reduces the error especially in conditions of high lateral engagement (lateral acceleration at 0.9 g) and low adhesion conditions (friction coefficient at 0.5) reducing the RMSE by about 0.05 km/h. Only in one case a worsening of the estimate is noted (steering wheel angle at 80 deg and vehicle speed at 100 km/h), but it can be seen that the difference is only 0.001 km/h.  Instead, about the side slip angle estimation, the advantages of this mix architecture are more visible. In fact, as shown in Figure 9, the UKF-ANN combination ensures in all the tests performed a RMSE A confirmation of these results is shown in Figures 10 and 11, where the results obtained in the longitudinal speed and side slip angle estimations by changing the steering angle between 40 deg, Figure  10a, 60 deg, Figure 10b, and 80 deg Figure 10c, and by changing the friction coefficient between 1, Figure 11a and 0.7 Figure 11b are presented. In these figures, it possible to see how the absolute error made by the UKF configuration grows with the time and with increasing steering angle and thus, keeping the longitudinal speed constant, with increasing lateral acceleration. Instead, the absolute error made by the UKF-ANN configuration is almost the same in the three dynamic layout.
In addition, when changing the road path conditions both the configuration increases a bit the absolute error but the UKF-ANN configuration maintains a lower one than the UKF configuration.

Conclusions
In conclusion, the current study was carried out with the intention of reduce the errors made by the state of art systems in the side slip angle estimation. Therefore, a mixed approach was investigated combining the potential of the Artificial Intelligence to approximate all the function without knowing a dynamic model, and the state observer capabilities of combining measure signal and dynamic model to reduce the estimation errors.
To test and validate the proposed architecture, a Montecarlo analysis was done by exploring the influence that some dynamic parameters have on the estimation process, such as longitudinal speed, lateral acceleration and friction coefficient. The results obtained show how the used of a ANN as sensor measure of the vehicle longitudinal and lateral velocities allows a reduction of the estimation errors in all the states tested up to a 10% error reduction considering a maximum side slip angle value of 6 deg, ensuring more robust estimation to vehicle dynamic changes. In this way, it would be possible to improved the performance of trajectory planning and tracking systems, as well as increased reliability of ESC stability systems implemented in autonomous, or standard cars.