{'fraction_non_alphanumeric': 0.03323304630494173, 'fraction_numerical': 0.006674274648130857, 'mean_word_length': 4.635938413564607, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Kalman Filter Meets Subjective Logic: A Self-Assessing Kalman Filter Using Subjective Logic

Self-assessment is a key to safety and robustness in automated driving. In order to design safer and more robust automated driving functions, the goal is to self-assess the performance of each module in a whole automated driving system. One crucial component in automated driving systems is the tracking of surrounding objects, where the Kalman filter is the most fundamental tracking algorithm. For Kalman filters, some classical online consistency measures exist for self-assessment, which are based on classical probability theory. However, these classical approaches lack the ability to measure the explicit statistical uncertainty within the self-assessment, which is an important quality measure, particularly, if only a small number of samples is available for the self-assessment. In this work, we propose a novel online self-assessment method using subjective logic, which is a modern extension of probabilistic logic that explicitly models the statistical uncertainty. Thus, by embedding classical Kalman filtering into subjective logic, our method additionally features an explicit measure for statistical uncertainty in the self-assessment.


I. INTRODUCTION
Being already widely used in the field of avionics and navigation [1], monitoring and assuring systems' functional performance have recently gained more and more importance for automated vehicles and is generally termed safety of the intended functionality (SOTIF) in the automotive context. Thus, self-assessment of the individual modules plays an important role to reach SOTIF; see, e.g., [2]. One crucial module in the perception of automated vehicles is the tracking of objects in its surrounding environment. For this task, the Kalman filter [3] is the most fundamental algorithm.
Classical approaches use the well-known normalized innovation squared (NIS) [4] for online self-assessment of Kalman filtering. The NIS monitors whether the Kalman filter's noise assumptions are consistent with the incoming measurements. In [5], Gibbs presents three tests to examine inconsistencies in Kalman filtering. The tests are designed to detect measurement outliers and model inconsistencies. Similar self-assessment quality measures have recently been used to adapt Kalman filter parameters depending on changing environments [6]- [8]. However, none of these works has taken into account the statistical uncertainty of the quality measure used for selfassessment. More precisely, the statistical uncertainty explicitly expresses the confidence of the quality measure itself. This type of uncertainty is typically called second-order probability. In fact, the statistical uncertainty can play an important role in self-assessment, particularly, if the number of samples is limited such that the quality measures may have limited statistical meaning. Then, using the statistical uncertainty, we are able to directly take into account how long the filter has already been consistent regarding the incoming measurements. This additional information can be further used to improve overall performance of the filter.
In this work, we present a novel approach to obtain a self-assessment measure in Kalman filtering using subjective logic; see Fig. 1. Subjective logic is a mathematical theory that explicitly models statistical uncertainty [9] similar to the Dempster-Shafer theory [10], [11]. Thus, our approach features a reliability measure that explicitly includes statistical uncertainty. This additional measure can be particularly beneficial if the number of samples is strictly limited, e.g., due to a fast-changing environment as we often observe in automated driving. Our proposed self-assessment method is able to online estimate the Kalman filter's performance and is presented as closed-form implementation in the theory of subjective logic.
Our contribution is two-fold: from a theoretical perspective, this work creates a never-before-seen link between subjective logic theory and Kalman filtering. From a practical perspective, we introduce a new online quality measure for selfassessment of Kalman filtering that additionally features a measure for the statistical uncertainty.
The remainder of this work is structured as follows. Section II describes similar works in the related field. In Section III, the fundamentals of subjective logic and Kalman filtering are summarized. Section IV presents our proposed method to obtain a self-assessment online measure for Kalman filtering using subjective logic. The simulation results of our proposed method are discussed in Section V. Finally, Section VI concludes our work.

II. RELATED WORK
The classical quality measure in Kalman filtering is the NIS [4]. Based upon the NIS and the normalized estimation error squared (NEES) [4], which needs, in contrast to the NIS, ground truth data, further consistency measures have been introduced in recent years. In [5], Gibbs presents three tests to examine inconsistencies in Kalman filtering. The smoother residual test and smother state test are derived, which are both based on a modified Bryson-Frazier smoother and are designed to detect measurement outliers and model inconsistencies, respectively. In addition, a filter residual test is introduced, which is also designed to detect measurement outliers. In [12], three equivalent derivations of the NIS and the resulting evaluation alternatives are presented. Firstly, the NIS is derived as a Bayesian p-test for the prior predictive distribution. Secondly, a derivation as a nested-model parameter significance test is given. Thirdly, a filter residual approach is described. In [13], a detailed evaluation of Kalman filtering is presented including indicators of, e.g., inner confidence, the determinant of the state transition matrix, properties of covariance matrices, and the Kalman gain.
Furthermore, Kalman filter tuning and adaptive Kalman filtering, which is often based on consistency measures, have gained some research attention in recent years. In [14], adaptive filtering for single target tracking is proposed, which selects appropriate filter algorithms depending on the NIS. Gelen et al. [6] develop three metrics to tune the Kalman filter in terms of process noise and measurement noise parameters. In [7], a method for auto-tuning Kalman filters with a Bayesian optimization strategy based on the NIS and NEES is designed. This method, however, needs ground truth data in order to use the NEES. Recently, Chen et al. [8] present how Bayesian optimization can resolve some issues in parameter tuning of Kalman filtering without having ground truth data.
In the context of temporal filtering and subjective logic, a subjective logic-based identification of Markov chains has been developed in [15]. The presented identification method generates, in addition to classical approaches, an explicit reliability measure in terms of statistical uncertainty of the identification result itself. Only slightly related is the approach ofŠkorić et al. [16]. They present evidence-based subjective logic as a combination of flow-based reputation systems with the uncertainty concept of subjective logic in order to determine indirect computational trust through a trust network. In fact, flow-based reputation systems have their mathematical foundation also in Markov chains.
However, to the best of our knowledge, neither the combination of subjective logic and Kalman filtering, nor the introduction of a self-assessment metric for Kalman filtering that explicitly includes a measure for the statistical uncertainty have been addressed in literature so far.

III. FUNDAMENTALS
This section summaries the mathematical foundation of subjective logic including some commonly used subjective logic operators, which are also required for our proposed method. In addition, we briefly summarize the Kalman filter and outline the consistency examination of Kalman filtering.

A. Subjective Logic
The mathematical description of subjective logic, which is summarized in the following, is mainly based on [9]. One key structure in subjective logic is the opinion representation. A multinomial opinion expresses information of a discrete random variable X in terms of belief, uncertainty, and base rate for every event x of the sample space X.
Definition 1 (Multinomial Opinion). Let X ∈ X be a random variable of the finite domain X with cardinality W = |X| ≥ 2. A multinomial opinion is an ordered triple Here, b X is the belief mass distribution over X, u X ∈ [0, 1] is the uncertainty mass representing the lack of evidence, and a X is the base rate distribution over X representing the prior probability. Moreover, the projected probability distribution of a multinomial opinion projects the opinion to a classical probability distribution and, thus, represents the expected outcome of an opinion in probability space.
To combine opinions from various sources about the same domain of interest, multiple fusion operators exist for merging these opinions. Generally speaking, this can be interpreted as a set of sources that come together in order to find a joint conclusion about a certain task using some fusion operator. For certain tasks, particular fusion operators are more suitable than others. Here, we present the aleatory cumulative belief fusion (A-CBF), which is an appropriate fusion operator for our method in Section IV. Further fusion operators can be found in [9]. Definition 2 (Aleatory Cumulative Belief Fusion). Let ω A X and ω B X be multinomial opinions of source A and B over the same variable X on domain X. Let ω A⋄B X be the fused opinion such that [9]. The opposite of fusion in subjective logic is called unfusion. The objective of an unfusion operator is to remove the input of a specific opinion from an already fused opinion. In fact, the unfusion operator of the A-CBF is called cumulative unfusion [9].
be the cumulative fused opinion as in (3) of ω B X and an unknown opinion ω A X over the variable X ∈ X with the same base rate of opinion B and C, namely a X . Let ω A X = ω C⋄B X be the unfused opinion such that X is called cumulative belief unfusion. For the special case u B X = u C X = 0, we refer to [9]. To obtain trust or belief from transitive trust paths, trust discounting is often used; for further details, please refer to [9]. We define and use trust discounting in a different way for our purpose.
Roughly speaking, trust discounting models that a certain amount of information will be lost while transferring this information via multiple sources. We use this subjective logic operator in our proposed method in the context of estimating time-varying parameters. More precisely, we use trust discounting to account for information degradation over time due to possible parameter changes.
Apart from fusion operators, a comparison operator called degree of conflict (DC) is defined in order to measure the difference between two opinions about the same variable X.
Definition 5 (Degree of Conflict). Let ω A X and ω B X be multinomial opinions of source A and B over the same variable X on domain X. Then, DC ω A X , ω B X denotes the degree of conflict between the two opinions ω A X and ω B X . The DC is defined as where For similar opinions, the DC is expected to be small, i.e., nearly zero, and for highly conflicting opinions, the DC is expected to be large, i.e., nearly the value of CC.

B. Kalman Filter
The Kalman filter [3] is an estimation algorithm for unknown variables based on a series of uncertain measurements.
The key assumptions of Kalman filtering are that all signals and probability densities are Gaussian distributed and the process and measurement models are linear. If these assumptions are fulfilled, then the Kalman filter is a Bayes-optimal state estimator [17] and facilitates a closed-form implementation of the Bayes filter for recursive state estimations.
The estimated state x k ∈ R n of an object at time step k ∈ N in Kalman filtering is modeled by an n-dimensional multivariate Gaussian distribution with meanx k ∈ R n and covariance matrix P k ∈ R n×n . The motion and measurement models are given by with the process matrix F k ∈ R n×n and the measurement matrix H k ∈ R m×n . The process noise v k ∈ R n and measurement noise w k ∈ R m are assumed to be uncorrelated and zero-mean Gaussian distributed. Then, the motion model in (7) yields to the predicted state of the object with the corresponding covariance matrix where Q k = E v k v T k ∈ R n×n is the covariance matrix of the process noise. The measurement prediction is stated bŷ where R k+1 = E w k+1 w T k+1 ∈ R m×m is the covariance matrix of the predicted measurement. Thus, the measurement matrix H k+1 displays the transformation from the state space into the measurement space. Typically, the measurement space is smaller than the state space, i.e. m < n, which means that not all components of the object state are measurable. The residual of the actual measurement z k+1 and the predicted measurementẑ k+1|k is defined as γ k+1 := z k+1 −ẑ k+1|k (13) and is used in the innovation of the Kalman filter. Then, the measurement z k+1 is taken into account during the update step yielding the posterior state estimation where the Kalman gain models the impact of the process and measurement model uncertainties towards the posterior state estimation. For small values of K k+1 , the posterior state estimation trusts more in the state predictionx k+1|k , i.e., the process model, and, accordingly, for big values of K k+1 , the posterior state estimation trusts more in the current measurement z k+1 .

C. Consistency of State Estimators
For estimating static parameters, consistency is defined such that the estimated value must converge with increasing number of measurements to the true value. For state estimation in dynamic systems, this consistency definition is not applicable due to the time-variant state. In [4], practical consistency conditions of state estimators are defined as where (17) depicts that the estimator should be unbiased and (18) describes that the mean square error should be equivalent to the estimated covariance matrix P k . For examining condition (18), which implicitly include (17), the NEES is used. The NEES follows a χ 2 distribution with n degrees of freedom (the dimension of the state space) if all assumptions of the Kalman filter are fulfilled. To check if the Kalman filter is consistent, the NEES must be in a certain confidence interval of the χ 2 distribution. However, to perform the NEES, a ground truth is necessary, which is often not available. For online applications the time-average NIS [18] is designed as a time-average value over a data window of size K ∈ N of the classical NIS ε γ k , which is the Mahalanobis distance of the measurement residual γ k with regard to the innovation covariance matrix S k . In fact, supposing ergodicity, if the Kalman filter's assumptions are fulfilled, then Kε γ also follows a χ 2 distribution with Km degrees of freedom.

IV. SELF-ASSESSMENT METHOD USING SUBJECTIVE LOGIC
In this section, starting with our problem formulation, we present our proposed algorithm for self-assessing Kalman filtering using subjective logic and explain the respective steps in detail.

A. Problem Formulation
Given a Kalman filter, the goal of the proposed method is to realize an online self-assessment of the Kalman filter's performance. Therefore, the proposed method monitors the validity of the statistical assumptions of Kalman filtering in online applications. This objective is similar to the NIS or time-average NIS if multiple measurements are used. However, in contrast to the traditional NIS, we want to use a measure for consistency testing that is more significant in terms of statistical evidence. Moreover, we want to generate a self-assessment online measure that also supplies an explicit certainty measure expressing the level of certainty about the statement. Hence, we can estimate the reliability of each sensor with respect to the filtering assumptions consisting of a self-assessment measure with an explicit certainty value of the measure.
Kalman filtering produces measurement prediction in terms ofẑ k+1|k ∈ R m and S k+1 ∈ R m×m for every time step k ∈ N, see (11) and (12), respectively. Using subjective logic and the incoming measurements z k+1 ∈ R m , the proposed method outputs a self-assessment online measure δ k ∈ [0, 1] and, additionally, a corresponding explicit uncertainty u δ k ∈ [0, 1] in every time step based on the filtering measurement predictions and assumptions. As our method typically uses multiple measurements, we compare our measure to the time-average NIS for ensuring a fair comparison.

B. Algorithm
The key idea of the algorithm is to form a multinomial opinion of the correctness of the Kalman filter's assumptions with respect to the incoming measurements. Consequently, we compare the generated opinion with an ideal Gaussian opinion based on the filtering assumptions. This comparison leads to a DC which gives us a self-assessment measure and a corresponding explicit uncertainty of this measure. Algorithm 1 portrays an overview of this procedure in order to determine a self-assessment online measure. As input, our proposed method needs a random variable X ∈ X = {x 1 , . . . , x nX }, n X ∈ N, which models the Gaussian distribution assumptions of Kalman filtering. This is implemented by discretizing the assumed Gaussian distribution in n X bins in order to use the evidence of our samples, i.e., the incoming measurements, in a supported subjective logic manner. Moreover, the initial opinion ω 0 X = (b 0 X , u 0 X , a X ) of the correctness of the Kalman filter's assumptions is constituted as vacuous opinions, i.e., u 0 X = 1 and b 0 X (x) = 0 ∀x ∈ X. In addition, a reference opinion ω G X = (b G X , u G X , a X ) of the assumed Gaussian distribution is featured as a dogmatic opinion, i.e., u 0 X = 0. Further, the number of time steps n ∈ N is specified. To be able to correctly monitor drifts and jumps in the ground truth noise parameters, Input: Random variable X ∈ X = {x1, . . . , xn X } with nX ∈ N modeling the assumptions for a Gaussian distribution, initial opinion ω 0 X = (b 0 X , u 0 X , aX ), reference opinion of the assumed Gaussian distribution ω G X = (b G X , u G X , aX ), number of time steps n ∈ N, window length nst ∈ N for short-term opinion generation, step size nc ∈ N with nc < nst for long-term and short-term opinion comparison, threshold θ ∈ [0, 1], trust discounting probability p d ∈ [0, 1] Output: Self-assessment online measure δ k ∈ [0, 1] of the correctness of the filtering assumptions with corresponding explicit uncertainty u δ k ∈ [0, 1] for k = 0, . . . , n 1: procedure SLCONSISTENCYMEASURE(X,ω 0 X , ω G X , n, nst, nc, θ, p d ) if k < nst − 1 then

5:
returnωX , ωz X 6: end procedure and a threshold θ ∈ [0, 1] for the corresponding comparison using the DC. For modeling the degradation of information for time-varying parameters over time, trust discounting is applied with respect to the probability p d ∈ [0, 1]. To neglect this aspect, the discount probability can be chosen to p d = 1.
After the initialization step, the first n st time steps are used to generate a short-term opinion about the correctness of the filter assumptions with respect to the incoming measurements. One important component, in doing so, is the procedure of updating the previous opinion with the incoming measurement. This procedure is displayed in Algorithm 2. In addition to the previous short-term opinion ω X , this procedure needs the Kalman filter's measurement predictionẑ ∈ R m , the covariance matrix S ∈ R m×m , and the incoming measurement z ∈ R m as input parameters. Then, the incoming measurement is mapped to a standard normal distribution based on the measurement prediction of Kalman filtering. The notation S −1/2 denotes the square root of the inverse of covariance matrix S, which can be obtained using, e.g., the Cholesky factorization. For further details, please refer to [18]. Consequently, the transformed measurementz is assigned to a certain event x i of X, i ∈ {1, . . . , n Z } such that a resulting opinion ωz X with respect to the measurement is generated. This opinion is fused with the previous opinion to generate the updated opinionω X . To conclude, the procedure returns the updated opinion and the generated opinion with respect to the measurement.
Continuing the procedure of Algorithm 1 and after processing the first n st time steps, we update the short-term and longterm opinion n c times while calculating the self-assessment measure in each time step. After n c time steps, we compare the short-term and long-term performance represented by opinions using the DC such that we are able to react quickly to sudden noise parameter changes, which are noticeable in the shortterm opinion. This is implemented such that, if the opinions match, i.e., the DC is smaller than a threshold, new opinions will be continuously merged with previous opinions, which is based on more statistical data and, hence, show less statistical uncertainty. On the downside, if the opinions do not match, i.e., the DC exceeds a certain threshold, the previous long-term opinion will be discarded. This procedure continues until time step n and is able to output the self-assessment online measure δ k and the corresponding uncertainty u δ k in every time step.

V. SIMULATION RESULTS
This section evaluates our proposed self-assessment method through simulated data. On the one hand, jumps and drifts in the ground truth measurement noise parameters are examined and, on the other hand, changes in the process model for the generation of ground truth data are evaluated.
For the following simulations, we consider a single-target multi-sensor simulation setup with two sensors measuring the position in one dimension of a single object in each time step. The two sensors are modeled to be equal in terms of Kalman filter's assumptions, i.e., the measurement noise is assumed to be w k ∼ N (0, σ w ) with constant variance σ w for both sensors. Moreover, we assume a constant velocity model with process noise v k ∼ N (0, σ v ). Further, it is assumed that σ w = σ v , which should describe the fact that we do not have prior knowledge about the noise parameters. The parameters of our proposed method are chosen in the following way. We choose n st = 35 to incorporate enough evidence to form a reliable short-term opinion, n c = 1 to be able to react quickly on parameter jumps, θ = 0.25 to define a threshold for the comparison of subjective opinions, and p d = 0.99 to apply trust discounting.

A. Jumps in Measurement Noise
We first consider jumps in our simulated ground truth measurement noise σ wgt . The progress of the ground truth measurement noise of our simulated sensors is illustrated in Fig. 2, where two jumps are located at time step 105 and 210. The underlying process model for the ground truth data generation and for the Kalman filter is a constant velocity model. The results of the first simulation scenario in terms of the time-average NIS are shown in Fig. 3. The 95% confidence interval of the time-average NIS is displayed as reference. It can be seen that the Kalman filter's assumptions are violated by these jumps in the simulated measurement noise during the corresponding time sections. With our proposed method, we obtain a self-assessment measure in Fig. 4(a) and the corresponding uncertainty in Fig. 4(b). We obtain similar results with the subjective logic-based measure as the time-average NIS in Fig. 3 distribution, i.e., the interval [0, ∞). In contrast, the subjective logic self-assessment measure is given as the DC between two opinions, i.e., as a normalized value in [0, 1]. Compared to the time-average NIS, our proposed self-assessment measure shows sharp edges when recognizing jumps and keeps the level of the measure more constant during the jumps. Particularly, the time-average NIS shows small collapses during the jumps.
In addition, the peaks in the uncertainty in Fig. 4(b) support the conclusion that our proposed method has recognized the jumps and has consequently discarded the long-term history.
Compared to the ground truth, these jumps are detected with small delays as well as for the time-average NIS. However this is plausible because in order to recognize jumps, and to be certain about it, a certain amount of statistical data has to be collected.

B. Drift in Measurement Noise
As second experiment, we consider a drift in our simulated ground truth measurement noise σ wgt , which is displayed in Fig. 5. Here, the ground truth measurement noise of sensor 1 drifts from start value of 1 meter to the end value of 3 meters. The underlying process model for the ground truth data generation and for the Kalman filter is again a constant velocity model. The results of the time-average NIS including the 95% confidence interval are illustrated in Fig. 6. Here, the measure of sensor 1 gets bigger as the simulated measurement noise gets bigger, while, the time-average NIS of sensor 2 levels off in the 95% confidence interval. The results of our proposed subjective logic-based method are visualized in Fig. 7. Our self-assessment measure needs approximately the first 40 time steps in order to clearly separate the performance of the two sensors, but afterwards, the drift is clearly monitored. This effect is reasonable as it can also be seen in Fig. 6 that approximately until time step 40 both time-average NIS values are within the 95% confidence interval. Our obtained uncertainty of the self-assessment measure is continuously decreasing, which supports the fact that we get more and more certain about our subjective logic-based measure with increasing time.

C. Changes in Process Model
As last experiment, we simulate changes in the underlying process model of our simulated ground truth data. The changes in our simulated velocity are displayed in Fig. 8. First, we consider a constant velocity of 35 meters per second, which matches our Kalman filter's assumptions of the process model  type. Then, the simulated velocity decreases down to the value of 5 meters per second which descriptively means that the target brakes. After a section with constant velocity of 5 meters per second, we accelerate again up to 35 meters per second. The calculated time-average NIS of this scenario is depicted in Fig. 9. In the sections of constant velocity, the consistency values are mostly within the 95% confidence interval. For the braking and acceleration sections, the consistency values of the two sensors are violated and outside of the confidence interval. Compared to the other experiments, the time-average NIS values are in general smaller, which results from a higher chosen process noise in the Kalman filter's assumptions in order to better visualize the important aspects of this scenario. The results of our proposed method are shown in Fig. 10. Compared to the time-average NIS, our self-assessment measure is again more consistent when considering the first braking phase. In Fig. 9, the time-average NIS has two peaks at   slightly different locations for sensor 1 and sensor 2. Actually, in our proposed method, this effect is also slightly visible by the peaks in the uncertainty at different time steps, but our self-assessment measure is generally smoother in this braking phase. Furthermore, our proposed self-assessment measure shows sharper edges when the velocity begins to decrease and increase. Additionally, the peaks in our uncertainty measure in Fig. 10(b) supports the recognition of the changes in the velocity progression as explained before.

VI. CONCLUSION
In this contribution, we proposed a self-assessment online method in Kalman filtering based on subjective logic theory. In contrast to classical consistency measures, such as the NIS, we are not only able to obtain a self-assessment online measure of the correctness of the Kalman filter's assumptions, but we are also able to obtain an explicit uncertainty. The latter states how certain we are about the calculated self-assessment measure. As evaluated through simulated data, our proposed method is able to compete with a time-average NIS approach and shows even superior results in some addressed aspects.
In our future work, we aim to implement an adaptive Kalman filter, which is based on our proposed online selfassessment algorithm. Due to the additionally obtained explicit uncertainty and the closed-form algorithm in subjective logic theory, we claim to be able to use subjective logic operators in order to obtain more accurate Kalman filter estimation results. Furthermore, we intend to investigate self-assessment of multitarget tracking algorithms using subjective logic.