Measuring the robustness of Gaussian processes to kernel choice
  Gaussian processes (GPs) are used to make medical and scientific decisions,
including in cardiac care and monitoring of atmospheric carbon dioxide levels.
Notably, the choice of GP kernel is often somewhat arbitrary. In particular,
uncountably many kernels typically align with qualitative prior knowledge
(e.g.\ function smoothness or stationarity). But in practice, data analysts
choose among a handful of convenient standard kernels (e.g.\ squared
exponential). In the present work, we ask: Would decisions made with a GP
differ under other, qualitatively interchangeable kernels? We show how to
answer this question by solving a constrained optimization problem over a
finite-dimensional space. We can then use standard optimizers to identify
substantive changes in relevant decisions made with a GP. We demonstrate in
both synthetic and real-world examples that decisions made with a GP can
exhibit non-robustness to kernel choice, even when prior draws are
qualitatively interchangeable to a user.
