Explaining Hyperparameter Optimization via Partial Dependence Plots
  Automated hyperparameter optimization (HPO) can support practitioners to
obtain peak performance in machine learning models. However, there is often a
lack of valuable insights into the effects of different hyperparameters on the
final model performance. This lack of explainability makes it difficult to
trust and understand the automated HPO process and its results. We suggest
using interpretable machine learning (IML) to gain insights from the
experimental data obtained during HPO with Bayesian optimization (BO). BO tends
to focus on promising regions with potential high-performance configurations
and thus induces a sampling bias. Hence, many IML techniques, such as the
partial dependence plot (PDP), carry the risk of generating biased
interpretations. By leveraging the posterior uncertainty of the BO surrogate
model, we introduce a variant of the PDP with estimated confidence bands. We
propose to partition the hyperparameter space to obtain more confident and
reliable PDPs in relevant sub-regions. In an experimental study, we provide
quantitative evidence for the increased quality of the PDPs within sub-regions.
