Uncertainty Quantification in Scientific Machine Learning: Methods,
  Metrics, and Comparisons
  Neural networks (NNs) are currently changing the computational paradigm on
how to combine data with mathematical laws in physics and engineering in a
profound way, tackling challenging inverse and ill-posed problems not solvable
with traditional methods. However, quantifying errors and uncertainties in
NN-based inference is more complicated than in traditional methods. This is
because in addition to aleatoric uncertainty associated with noisy data, there
is also uncertainty due to limited data, but also due to NN hyperparameters,
overparametrization, optimization and sampling errors as well as model
misspecification. Although there are some recent works on uncertainty
quantification (UQ) in NNs, there is no systematic investigation of suitable
methods towards quantifying the total uncertainty effectively and efficiently
even for function approximation, and there is even less work on solving partial
differential equations and learning operator mappings between
infinite-dimensional function spaces using NNs. In this work, we present a
comprehensive framework that includes uncertainty modeling, new and existing
solution methods, as well as evaluation metrics and post-hoc improvement
approaches. To demonstrate the applicability and reliability of our framework,
we present an extensive comparative study in which various methods are tested
on prototype problems, including problems with mixed input-output data, and
stochastic problems in high dimensions. In the Appendix, we include a
comprehensive description of all the UQ methods employed, which we will make
available as open-source library of all codes included in this framework.
