Model-Constrained Deep Learning Approaches for Inverse Problems
  Deep Learning (DL), in particular deep neural networks (DNN), by design is
purely data-driven and in general does not require physics. This is the
strength of DL but also one of its key limitations when applied to science and
engineering problems in which underlying physical properties (such as
stability, conservation, and positivity) and desired accuracy need to be
achieved. DL methods in their original forms are not capable of respecting the
underlying mathematical models or achieving desired accuracy even in big-data
regimes. On the other hand, many data-driven science and engineering problems,
such as inverse problems, typically have limited experimental or observational
data, and DL would overfit the data in this case. Leveraging information
encoded in the underlying mathematical models, we argue, not only compensates
missing information in low data regimes but also provides opportunities to
equip DL methods with the underlying physics and hence obtaining higher
accuracy. This short communication introduces several model-constrained DL
approaches (including both feed-forward DNN and autoencoders) that are capable
of learning not only information hidden in the training data but also in the
underlying mathematical models to solve inverse problems. We present and
provide intuitions for our formulations for general nonlinear problems. For
linear inverse problems and linear networks, the first order optimality
conditions show that our model-constrained DL approaches can learn information
encoded in the underlying mathematical models, and thus can produce consistent
or equivalent inverse solutions, while naive purely data-based counterparts
cannot.
