Deep Multi-Fidelity Active Learning of High-dimensional Outputs
  Many applications, such as in physical simulation and engineering design,
demand we estimate functions with high-dimensional outputs. The training
examples can be collected with different fidelities to allow a cost/accuracy
trade-off. In this paper, we consider the active learning task that identifies
both the fidelity and input to query new training examples so as to achieve the
best benefit-cost ratio. To this end, we propose DMFAL, a Deep Multi-Fidelity
Active Learning approach. We first develop a deep neural network-based
multi-fidelity model for learning with high-dimensional outputs, which can
flexibly, efficiently capture all kinds of complex relationships across the
outputs and fidelities to improve prediction. We then propose a mutual
information-based acquisition function that extends the predictive entropy
principle. To overcome the computational challenges caused by large output
dimensions, we use multi-variate Delta's method and moment-matching to estimate
the output posterior, and Weinstein-Aronszajn identity to calculate and
optimize the acquisition function. The computation is tractable, reliable and
efficient. We show the advantage of our method in several applications of
computational physics and engineering design.
