Preference Exploration for Efficient Bayesian Optimization with Multiple
  Outcomes
  We consider Bayesian optimization of expensive-to-evaluate experiments that
generate vector-valued outcomes over which a decision-maker (DM) has
preferences. These preferences are encoded by a utility function that is not
known in closed form but can be estimated by asking the DM to express
preferences over pairs of outcome vectors. To address this problem, we develop
Bayesian optimization with preference exploration, a novel framework that
alternates between interactive real-time preference learning with the DM via
pairwise comparisons between outcomes, and Bayesian optimization with a learned
compositional model of DM utility and outcomes. Within this framework, we
propose preference exploration strategies specifically designed for this task,
and demonstrate their performance via extensive simulation studies.
