Robust Expected Information Gain for Optimal Bayesian Experimental Design Using Ambiguity Sets
The ranking of experiments by expected information gain (EIG) in Bayesian
experimental design is sensitive to changes in the model's prior distribution,
and the approximation of EIG yielded by sampling will have errors similar to
the use of a perturbed prior. We define and analyze \emph{robust expected
information gain} (REIG), a modification of the objective in EIG maximization
by minimizing an affine relaxation of EIG over an ambiguity set of
distributions that are close to the original prior in KL-divergence. We show
that, when combined with a sampling-based approach to estimating EIG, REIG
corresponds to a `log-sum-exp' stabilization of the samples used to estimate
EIG, meaning that it can be efficiently implemented in practice. Numerical
tests combining REIG with variational nested Monte Carlo (VNMC), adaptive
contrastive estimation (ACE) and mutual information neural estimation (MINE)
suggest that in practice REIG also compensates for the variability of
under-sampled estimators.