Transfer Learning on Multi-Fidelity Data
  Neural networks (NNs) are often used as surrogates or emulators of partial
differential equations (PDEs) that describe the dynamics of complex systems. A
virtually negligible computational cost of such surrogates renders them an
attractive tool for ensemble-based computation, which requires a large number
of repeated PDE solves. Since the latter are also needed to generate sufficient
data for NN training, the usefulness of NN-based surrogates hinges on the
balance between the training cost and the computational gain stemming from
their deployment. We rely on multi-fidelity simulations to reduce the cost of
data generation for subsequent training of a deep convolutional NN (CNN) using
transfer learning. High- and low-fidelity images are generated by solving PDEs
on fine and coarse meshes, respectively. We use theoretical results for
multilevel Monte Carlo to guide our choice of the numbers of images of each
kind. We demonstrate the performance of this multi-fidelity training strategy
on the problem of estimation of the distribution of a quantity of interest,
whose dynamics is governed by a system of nonlinear PDEs (parabolic PDEs of
multi-phase flow in heterogeneous porous media) with uncertain/random
parameters. Our numerical experiments demonstrate that a mixture of a
comparatively large number of low-fidelity data and smaller numbers of high-
and low-fidelity data provides an optimal balance of computational speed-up and
prediction accuracy. The former is reported relative to both CNN training on
high-fidelity images only and Monte Carlo solution of the PDEs. The latter is
expressed in terms of both the Wasserstein distance and the Kullback-Leibler
divergence.
