A Lagrangian Dual-based Theory-guided Deep Neural Network
  The theory-guided neural network (TgNN) is a kind of method which improves
the effectiveness and efficiency of neural network architectures by
incorporating scientific knowledge or physical information. Despite its great
success, the theory-guided (deep) neural network possesses certain limits when
maintaining a tradeoff between training data and domain knowledge during the
training process. In this paper, the Lagrangian dual-based TgNN (TgNN-LD) is
proposed to improve the effectiveness of TgNN. We convert the original loss
function into a constrained form with fewer items, in which partial
differential equations (PDEs), engineering controls (ECs), and expert knowledge
(EK) are regarded as constraints, with one Lagrangian variable per constraint.
These Lagrangian variables are incorporated to achieve an equitable tradeoff
between observation data and corresponding constraints, in order to improve
prediction accuracy, and conserve time and computational resources adjusted by
an ad-hoc procedure. To investigate the performance of the proposed method, the
original TgNN model with a set of optimized weight values adjusted by ad-hoc
procedures is compared on a subsurface flow problem, with their L2 error, R
square (R2), and computational time being analyzed. Experimental results
demonstrate the superiority of the Lagrangian dual-based TgNN.
