{'fraction_non_alphanumeric': 0.04146255437551436, 'fraction_numerical': 0.011012266332249089, 'mean_word_length': 4.510256963938674, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Modeling the decision-making in human driver overtaking ⋆

We propose models for the decision-making process of human drivers in an overtaking scenario. First, we mathematically formalize the overtaking problem as a decision problem with perceptual uncertainty. Then, we propose and numerically analyze risk-agnostic and risk-aware decision models, which are able to judge whether an overtaking is desirable or not. We show how a driver’s decision-making time and confidence level can be primarily characterized through two model parameters, which collectively represent human risk-taking behavior. We detail an experimental testbed for evaluating the decision-making process in the overtaking scenario. Finally, we present some preliminary experimental results from two human drivers.


Motivation
Transportation networks face the mixed-autonomy challenge wherein automated vehicles, with various degrees of autonomy, are gradually being introduced into road traffic: Lazar et al. (2017); Mehr and Horowitz (2019); Wei et al. (2019). For instance, the first full-sized automated bus will begin to operate in Singapore in conjunction with regular traffic this year, see Wei (2019), and automated minibuses already drive regularly in confined areas in Stockholm: Löfgren (2017). In these mixed traffic environments, there are two types of decision-makers: human drivers and automated vehicles. The safety-level is determined by the interplay between these two types of agents.
The design of decision strategies for automated vehicles requires a thorough understanding of human driver decisionmaking processes. Here, it is important not just to predict what the human will do, but to find the underlying mechanisms why the human makes the decision. Obtaining these key mechanisms is important in its own right, but can also improve decision-making policies for automated vehicles (e.g., enhancing and guiding model-free human prediction models) and to advance driver assistance systems (such as alerting the human driver when an unsafe mode of driving occurs).
Motivated by these observations, this paper considers the decision-making process of human drivers in overtaking scenarios, one of the most dangerous scenarios drivers can encounter, e.g., Hegeman et al. (2004); Wang and Knipling (1994); Barr and Najm (2001). Overtaking scenarios are, from an experimental point of view, rich in the sense that the human is making decisions in a dynamic environment under limited time and subject to risk.

Contribution
We study the decision-making process of a human driver in an overtaking scenario as illustrated in Fig. 1. Our contributions are three-fold: (1) We introduce a mathematical formulation of the overtaking problem. In this formulation, an ego vehicle E driven by a human is on a two-way road stuck behind a slow-moving vehicle O, and with an approaching vehicle M in the other direction. E hence has a choice: overtake or wait. We assume E has perfect perception of O, but sequentially receives noisy measurements of M , since M is supposed to be further away.
(2) We propose decision models judging whether E should overtake or not. Each decision model consists of an estimator and a decision rule to overtake or wait based on the current estimate. We consider two decision rules: one risk-agnostic rule that assumes the estimate to be groundtruth, granting an overtaking if the estimate indicates a safe overtaking, and one risk-aware rule that only grants an overtaking if it is confident enough that an overtaking is safe (based on the confidence of the estimate). We analyze the behaviors emerging from these decision models.
(3) We introduce an experimental testbed where a human driver performs overtaking maneuvers in a driving simulator. We explain the setup of the experiment and present some preliminary results.

Related work
Substantial efforts have been dedicated recently to study and improve the decision-making process of automated vehicles, see Brechtel et al. (2014); Furda and Vlacic (2011);Cunningham et al. (2015). Here, the mixed traffic condition has attracted the use of sophisticated human prediction models for guiding the automated vehicle, e.g., Ziebart et al. (2009);Sadigh et al. (2016); Stefansson et al. (2019). Even though the expressiveness of these models varies, the primary focus is on assisting the automated vehicle by predicting how the human will behave, and not on understanding the underlying mechanisms yielding the human decision. Hence, the assumptions made (e.g., the human has perfect information or knows the automated vehicle's objective) are often left unconfirmed from a human decision-making perspective.
Extensive work has been conducted in the field of psychology to understand the decision-making process of humans. From a psychological perspective, the overtaking scenario involves a choice between a certain undesirable alternative (stay) and a risky alternative (overtake) with both a desirable outcome (successful) and a highly unwanted outcome (unsuccessful). Such choice situations have long been the fruit fly of behavioral decision-making research with numerous systematic behaviors observed, e.g., a general aversion to risky or uncertain alternatives, see Oppenheimer and Kelso (2015), and related models such as Prospect Theory and Decision Field Theory, see Kahneman and Tversky (1979), Busemeyer and Townsend (1993). Previously, human behavior in overtaking scenarios has been studied by statistical analysis of experimental results, see Regan (2000, 2005); Gordon and Mast (1970). In contrast, we provide a mathematical model for human decision-making in overtaking scenarios based on risk. Finally, the human perception of our model is based on optimal use of the observed information leading to a Kalman filter formulation. Speekenbrink and Shanks (2010) validate human prediction models in dynamic environments based on Bayesian filters, but focus on noninteractive stock market predictions and not on driving scenarios.

Outline
The paper is organized as follows. The mathematical formulation of the overtaking problem is described in Section 2. Our decision models for the overtaking problem are presented in Section 3. A numerical study of our decision models is performed in Section 4, followed by the description of our experiment and its results in Section 5. Section 6 concludes the paper. Solutions to this overtaking problem are proposed in the next section via decision models judging whether an overtaking is desirable or not. The decision models all make the following standing assumptions: The dynamics are modelled as a discrete-time system indexed by t ∈ N (start time t = 0) with state s(t) = [∆x(t), ∆v(t)] T , where ∆x(t) is the longitudinal distance between E and M , and ∆v(t) := v o − v m is the corresponding velocity difference. The state evolves with time step ∆t > 0 according to s(t + 1) = F s(t) where F = [1, −∆t; 0, 1]. Furthermore, in accordance with Assumption (iii), at each time t, E obtains noisy measurements y(t) = s(t) + η(t), where η(t) ∼ N (0, R(t)) is Gaussian noise with zero mean and covariance R(t).

DECISION MODELS
This section considers decision models for the overtaking problem. The considered decision models for the ego vehicle E have the structure in Fig. 2. At each time step t, the estimator, potentially initialized by a prior, is updated with a new (perceptual) measurement of the environment. The updated estimate is then passed to a decision rule which evaluates the decision based on the current estimate. The process ends with an action being taken (overtake or wait).

Estimator
The estimator assumes that E takes into account the obtained perceptual measurements {y(0), . . . , y(t)}, and uses this information optimally to obtain an estimate of the current state s(t). Mathematically, letŝ t|t ′ denote the estimate at time t given measurements {y (0) its corresponding covariance error. Our optimality assumption then states that the estimatorŝ t|t should minimize the covariance error min For a given initial distribution s(0) ∼ N (µ, Σ), the Kalman filter estimateŝ t|t minimizes (1), obtained recursively via the Kalman filter equations with initializationŝ 0|−1 = µ and P 0|−1 = Σ. In practice, the true distribution s(0) ∼ N (µ, Σ) is unknown to E. The estimator can then instead have an internal guess (µ 0 , Σ 0 ) of (µ, Σ), initializing the Kalman filter withŝ 0|−1 = µ 0 and P 0|−1 = Σ 0 , i.e., using the proxy s(0) ∼ N (µ 0 , Σ 0 ). For brevity, we then say that the Kalman filter is initialized with prior (µ 0 , Σ 0 ). Alternatively, the estimator does not have a prior (µ 0 , Σ 0 ), but, for instance, estimates s(0) with the first measurements. One such case we consider is the initializationŝ 0|0 = y(0) and P 0|0 = R. We then say that the Kalman filter is initialized without prior. In both cases, subsequent estimates are obtained via the Kalman filter equations.
An important aspect with having a prior (µ 0 , Σ 0 ) is that it enables us to capture a priori belief of E, such as bias in the decision model. Since human actions typically depend on past experiences (e.g., the human has learned), the prior serves as a tool to model such features. As an example, an experienced driver might have a very strong prior that the meeting car M will appear right after another meeting vehicle due to previous encounters of similar situations, while an unexperienced driver might exclusively focus on what she sees having a very uncertain prior. In Section 4, we systematically investigate how such priors can affect the overtaking decision.
We next consider our decision rules: the risk-agnostic and the risk-aware decision rules.

Risk-agnostic decision rule
The risk-agnostic decision rule obtains at each time t an estimateT m (t) of the time T m (t) when E would meet M during overtaking, and compares it with the time T o it would take E to overtake O. It then overtakes if T o <T m (t). More precisely, assuming E needs to cover a relative distance ∆x o when overtaking O, holding a overtaking velocity v to > v o , we get Here, we have assumed that E can change lanes momentarily and velocity momentarily from v o to v to and that v to is constant throughout the overtaking, yielding a constant overtaking time T o . Furthermore, The decision rule is then Risk-aware decision rule The risk-agnostic decision rule overtakes whenever T o <T m (t), independent of the uncertainty inT m (t). Thus, the decision rule is agnostic to risk and, hence, is potentially unsafe. To account for the uncertainty in the current estimate, the risk-aware decision rule overtakes only when the probability of the event T o < T m (t) is high, conditioned on the current estimatê s t|t . More precisely, the decision rule is (2) Here, 0 ≤ δ ≤ 1 is the confidence level of the decision, and β ≥ 0 prescribes an additional time margin for the overtaking.
Define the constant row vector and scalar L : The computation process for (2) is then: (1) Obtain jointly Gaussian distribution of [Ls(t),ŝ t|t ].
(2) Obtain conditional Gaussian distribution Ls(t)|ŝ t|t . ( Under the assumption that the Kalman filter is initialized with prior (µ, Σ), the conditional Gaussian distribution Ls(t)|ŝ t|t is equal to Ls(t)|y 0 , . . . , y t . Therefore, the confidence in (2) is equal to c t := P(T o + β < T m (t) |ŝ t|t ) = P(T o + β < T m (t) | y 0 , . . . , y t ), hence optimal, since there is no performance loss working witĥ s t|t instead of the measurements. One may also ask if the decision rule is optimal (under certain conditions) given the current confidence c t . The answer is yes. To see it, abbreviate the events "overtaking successful", "overtaking not successful", and "not overtaking" by OS, ON S, N O, respectively. Let u E be the utility of E defined by the triple (u E (OS), u E (ON S), u E (N O)) and E[u E |a] denote expected utility with respect to c t given action a ∈ {Overtake, Wait}. That is, (3) It is easy to see that (2) coincides with (3) provided Hence, (2) is in this sense optimal being equivalent to the solution of an expected utility maximization.

NUMERICAL EVALUATIONS
In this section, we numerically investigate the performance of the decision models. We start with the risk-agnostic decision rule, showcasing its shortcomings, and then continue with the risk-aware decision rule illustrating how the parameters in the model affect the outcome, in particular having a prior.

Risk-agnostic decision rule
We combine the risk-agnostic decision rule with an estimator with no prior. The parameters used are found in Table 1, assuming, for simplicity, constant noise covariance R(t) ≡ R. To get some intuition, we plot the probability P(T o <T m (t)) that this decision model grants an overtaking 2 as a function of time (start time zero), seen in Fig. 3, together with the correct decision 1{T o < T m (t)}. If the overtaking probability is lower than the correct decision (i.e., the decision model sometimes does not grant overtaking when it should), then we have a performance loss, while if the overtaking probability is higher than the correct decision (i.e., the decision model sometimes grant overtaking when it should not), then we have a safety loss. Since the rule is risk-agnostic, it should not differentiate between these losses and we hence suspect them to be within the same order of magnitude. Looking at Fig. 3, we see that this is also the case. In particular, the safety loss is a major part of the deviation, which is undesirable. The safety-loss can also be illustrated in an "available time-decision time"-diagram where the decision time is the first time the decision rule grants an overtaking and the available time is the time when M passes E (if no overtaking). We vary the available time by setting different initial distances between E and M . Running 50 realizations for each time t ∈ {4, 4 + ∆t, 4 + 2∆t, . . . , 15 − ∆t, 15}, we obtain a scatter plot seen in Fig. 4, where the red line marks the border between safe overtakings (T o < T m (t)) and unsafe overtakings (T o > T m (t)). The risk-agnostic feature is also here present with a large part of undesired, unsafe overtakings for small available times.

Risk-aware decision rule
We start by investigating the risk-aware decision rule with no prior in the estimator and then continue with the prior case. The setup in Table 1 is used throughout.
Decision rule without prior We start by illustrating the typical behavior of the confidence P(T o + β < T m (t) |ŝ t|t ) as a function of time t, noting that it fully determines the output of the decision rule (2). One expects the confidence to have a "low-high-low" trend due to few measurements in the beginning (but large amounts of time) and no time in the end (though many measurements). This is also what we generally get, with example in Fig 5a for δ = 0.1 and β = 0. Here, the black line is the average confidence over 1000 realizations while the grey area denotes one standard deviation, and the correct decision 1{T o < T m (t)}, the confidence threshold 1 − δ (see (2)), and the desired decision 1{T o + β < T m (t)} are plotted in red, dashed green, and dashed orange, respectively. We note that varying δ simply corresponds to vertically changing the threshold 1 − δ in dashed green. With δ = 0.1, this threshold is low and the rule quickly grants overtakings. The feature is also apparent in a corresponding scatter plot seen in Fig. 5b, with almost immediate decision times and a small portion of unsafe overtakings for small available times. Decreasing δ and thus increasing the threshold 1−δ (increasing the dashed green line in Fig. 5a) results in more risk-averse behavior with longer decision times and less granted overtakings for small available times, as seen in Fig. 5c with δ = 10 −4 and β = 0. In particular, note the increasing trend in decision time for decreased available time, due to the fact that a smaller available time needs more measurements to yield a high confidence.
So far, we set β = 0. Adding a time margin β > 0 is equivalent to decreasing the available time by β seconds. In a plot like Fig. 5a, adding β > 0 shifts 1{T o + β < T m (t)} horizontally to the left by β relative to the correct decision, with an example in Fig. 6a for δ = 0.01 and β = 3. Here, the decision rule has less time to make a decision (having the dashed orange line as the new time limit) with lower confidence as a result. Also, the variation is larger since the first measurements play a more crucial role for estimating if an overtaking is desirable or not. In a scatter plot, the addition of β just shifts the distribution to the right (by β) relative to the case β = 0.
Decision rule with prior We now consider the risk-aware decision rule and an estimator with a prior (µ 0 , Σ 0 ), fixing δ = 0.01 and β = 0 throughout. Assuming first a correct mean in the prior, µ 0 = s(0), a very low Σ 0 results in a confidence following the correct decision 1{T o < T m (t)} closely (due to an accurate initial estimate of s(0)). Higher values of Σ 0 yields more uncertain confidence, though still more certain than having an estimator with no prior, as The difference is most apparent in the beginning of Fig. 6b (0-1 s), where the confidence with a prior is higher with lower variation (due to initial guess of s(0)) than the noprior case (having no initial guess). For later times, the two confidences asymptotically approach similar shapes, as the measurements become dominant. In the limit Σ 0 → ∞, the two cases become identical.
Consider now the case when we instead have a bias: µ 0 = s(0). Assume first a pessimistic bias expecting M to be closer than the true value. A result with µ 0 = [0, ∆v(t)] T and Σ 0 as in (4) is given in Fig. 6c. Here, the decision model is initially unsure whether an overtaking is desired due to the pessimistic prior, but as measurements accumulate, it approaches the no-prior decision model. The delay in high confidence is also seen in a scatter plot depicted in Fig. 7a.
Finally, we conclude with an optimistic bias µ 0 = [2∆x(0), ∆v(t)] T and Σ 0 as in (4). The result is seen in Fig. 7b. The decision model is initially highly certain that an overtaking is desirable, which results in occasional unsafe overtakings seen in Fig. 7c for small available times.

Summary
The risk-agnostic decision rule is subject to unsafe overtaking behavior due to its risk-agnostic feature. The riskaware decision rule has safer overtaking with varying behavior depending on the parameters δ, β and potential prior (µ 0 , Σ 0 ): lower δ yields more certain overtakings with longer decision times, especially for small available times (see Fig. 5c); higher β increases the extra marginal time in overtakings corresponding to horizontally shifted distributions in the scatter plots. Finally, the risk-aware decision rule having a prior (µ 0 , Σ 0 ) with pessimistic bias yields longer overtaking times due to initial mismatch while an optimistic bias makes more rush decision with occasional unsafe overtakings for short available times.

EXPERIMENTAL EVALUATION
In this section, we outline the experimental setup that we developed and designed for collecting data on human 3 That is, 100 meter standard deviation in ∆x (0) Fig. 9. The experimental road map with an initial warm up region and 11 possible regions to setup an overtaking scenario. overtaking decision-making processes and present preliminary results from experimental trials on real humans. This experimental setup will be used for an experimental campaign, discussed further in Section 6.

Experimental Setup
The aim of our experimental setup is to provide a realistic environment in which human drivers end up in scenarios where they need to decide whether to overtake a slow vehicle or not.
For our driving simulator, we built a custom driving environment and API on top of the CARLA simulator (Dosovitskiy et al. (2017)). We interfaced a Logitech G29 steering wheel and pedals with the simulator. We position the human driver's perspective in the simulator relative to the steering wheel such that the driver feels that she is in the cabin of the simulated vehicle. An example of a human driver interacting with the simulator is shown in Fig. 8.
At the beginning of each trial, we first introduce the subject to the simulator and the task we would like her to complete. Then, we allow her to use the simulator for 15 minutes while we record several data points on her interactions.
In Fig. 9, we show the full route of the experiment. The experiment starts with 2 minutes of warm-up where the subject gets used to the controls of the simulator and encounters vehicles in the oncoming direction. Then, the subject begins to enter the different stages of the experiment. In each stage, the subject (E in Section 2) encounters the following entities: • a slow vehicle (O in Section 2) in front of the subject in the same lane, which the subject should try to overtake; • a line of blocking vehicles in the oncoming direction, which should prevent the subject from overtaking the slow vehicle until the subject is tailing the slow vehicle; 4 and • a risk vehicle (M in Section 2) following the line of blocking vehicles at a random distance (serves as our random initial condition in a scenario). Fig. 10. A series of snapshots from an example overtaking by a human driver. In the first snapshot, we can see the blocking line of vehicles on the left. In the second and third snapshots the slow vehicle is approached and the overtaking maneuver has begun. In the final snapshot, we can see the risk vehicle on the left.
The scenario set up on each stage is illustrated in Fig. 10.
There are a total of 11 stages. For each stage, we measure the positions and velocities of each vehicle. Furthermore, we measure the lane crossings of the ego vehicle; these time stamps are used to determine when the human driver made her overtaking decision.

Preliminary Results
We present two illustrative human driver results. The first human driver is an individual with prior knowledge of the experiment, called Driver A. The second human driver is an individual with no prior knowledge of the experiment, called Driver B. For both drivers, we present their cumulative driving data over all of their interactions with the driving simulator, in Fig. 11. We visualize each of the data sets using the same "available time-decision time"-diagram used in Section 4. We collected data on a total of 72 (60) minutes of driving time and 27 (25) overtaking decisions from Driver A (B). We plot the results in red (blue) in Fig. 11, where we observe the following characteristics about the two driver's data sets: 5 • both Driver A and Driver B never make an unsafe overtaking; • as available time increases, both Driver A and Driver B take increasingly more varied time to make a decision; however, Driver B exhibits a smaller spread; • the mean decision time remains fairly constant as available time increases.
We find that the two data sets exhibit phenomenon that are not completely captured by our decision models. In particular, there is more variance in the experimental data at larger available times than in the numerical results. Also, the fairly constant mean decision time indicates that the human decision process may not only involve continuous updates of core variables, but also rely on some all-or-nothing behaviors. However, to determine whether these are real phenomena, we will need to collect a more comprehensive data set. As discussed in the concluding section, we will begin an experimental campaign to obtain this statistically meaningful data set to further investigate the validity of our models.

CONCLUSION
This paper considered the decision-making process of human drivers in overtaking scenarios. The overtaking problem was mathematically formalised as a decision problem for the ego vehicle (possibly driven by a human) with perceptual uncertainty concerning an approaching vehicle in the opposite lane. Based on this formulation, we proposed decision models judging whether an overtaking is desirable or not. Each decision model is composed by an estimator and a decision rule, where the latter makes the decision based on an estimate from the former. The estimator is implemented as a Kalman filter, and two decision rules were considered: one risk-agnostic rule that assumes the estimate to be ground-truth, granting an overtaking if the estimate indicates a safe overtaking, and one risk-aware rule that only grants an overtaking when its confident enough that an overtaking is safe (based on the confidence of the estimate). We also presented a comprehensive analysis illustrating how the parameters in the decision models results in different overtaking behavior. Furthermore, we presented a realistic experimental testbed that we use as an evaluation platform for human driving overtaking. Finally, we presented some preliminary results where we begin to see the type of data our experimental platform yields.
Future work includes investigating other estimators (e.g., particle filters), and see how the covariance R(t) can be obtained from perceptual studies in psychology. We will also start an experimental campaign to obtain a statistically significant driving behavior data set and see how the result relates to the current decision models; in particular, we will estimate β and δ in the risk-aware decision rule from the obtained data set and see how well the parameters reproduce the result.