From Spoken Thoughts to Automated Driving Commentary: Predicting and
  Explaining Intelligent Vehicles' Actions
  In commentary driving, drivers verbalise their observations, assessments and
intentions. By speaking out their thoughts, both learning and expert drivers
are able to create a better understanding and awareness of their surroundings.
In the intelligent vehicle context, automated driving commentary can provide
intelligible explanations about driving actions, thereby assisting a driver or
an end-user during driving operations in challenging and safety-critical
scenarios. In this paper, we conducted a field study in which we deployed a
research vehicle in an urban environment to obtain data. While collecting
sensor data of the vehicle's surroundings, we obtained driving commentary from
a driving instructor using the think-aloud protocol. We analysed the driving
commentary and uncovered an explanation style; the driver first announces his
observations, announces his plans, and then makes general remarks. He also
makes counterfactual comments. We successfully demonstrated how factual and
counterfactual natural language explanations that follow this style could be
automatically generated using a transparent tree-based approach. Generated
explanations for longitudinal actions (e.g., stop and move) were deemed more
intelligible and plausible by human judges compared to lateral actions, such as
lane changes. We discussed how our approach can be built on in the future to
realise more robust and effective explainability for driver assistance as well
as partial and conditional automation of driving functions.
