XAutoML: A Visual Analytics Tool for Establishing Trust in Automated
  Machine Learning
  In the last ten years, various automated machine learning (AutoML) systems
have been proposed to build end-to-end machine learning (ML) pipelines with
minimal human interaction. Even though such automatically synthesized ML
pipelines are able to achieve a competitive performance, recent studies have
shown that users do not trust models constructed by AutoML due to missing
transparency of AutoML systems and missing explanations for the constructed ML
pipelines. In a requirements analysis study with 26 domain experts, data
scientists, and AutoML researchers from different professions with vastly
different expertise in ML, we collect detailed informational needs to establish
trust in AutoML. We propose XAutoML, an interactive visual analytics tool for
explaining arbitrary AutoML optimization procedures and ML pipelines
constructed by AutoML. XAutoML combines interactive visualizations with
established techniques from explainable artificial intelligence (XAI) to make
the complete AutoML procedure transparent and explainable. By integrating
XAutoML with JupyterLab, experienced users can extend the visual analytics with
ad-hoc visualizations based on information extracted from XAutoML. We validate
our approach in a user study with the same diverse user group from the
requirements analysis. All participants were able to extract useful information
from XAutoML, leading to a significantly increased trust in ML pipelines
produced by AutoML and the AutoML optimization itself.
