Multi-fidelity regression using artificial neural networks: efficient
  approximation of parameter-dependent output quantities
  Highly accurate numerical or physical experiments are often time-consuming or
expensive to obtain. When time or budget restrictions prohibit the generation
of additional data, the amount of available samples may be too limited to
provide satisfactory model results. Multi-fidelity methods deal with such
problems by incorporating information from other sources, which are ideally
well-correlated with the high-fidelity data, but can be obtained at a lower
cost. By leveraging correlations between different data sets, multi-fidelity
methods often yield superior generalization when compared to models based
solely on a small amount of high-fidelity data. In this work, we present the
use of artificial neural networks applied to multi-fidelity regression
problems. By elaborating a few existing approaches, we propose new neural
network architectures for multi-fidelity regression. The introduced models are
compared against a traditional multi-fidelity scheme, co-kriging. A collection
of artificial benchmarks are presented to measure the performance of the
analyzed models. The results show that cross-validation in combination with
Bayesian optimization consistently leads to neural network models that
outperform the co-kriging scheme. Additionally, we show an application of
multi-fidelity regression to an engineering problem. The propagation of a
pressure wave into an acoustic horn with parametrized shape and frequency is
considered, and the index of reflection intensity is approximated using the
multi-fidelity models. A finite element model and a reduced basis model are
adopted as the high- and low-fidelity, respectively. It is shown that the
multi-fidelity neural network returns outputs that achieve a comparable
accuracy to those from the expensive, full-order model, using only very few
full-order evaluations combined with a larger amount of inaccurate but cheap
evaluations of a reduced order model.
