Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for
  Dynamic Control
  Model-based reinforcement learning (MBRL) is believed to have much higher
sample efficiency compared to model-free algorithms by learning a predictive
model of the environment. However, the performance of MBRL highly relies on the
quality of the learned model, which is usually built in a black-box manner and
may have poor predictive accuracy outside of the data distribution. The
deficiencies of the learned model may prevent the policy from being fully
optimized. Although some uncertainty analysis-based remedies have been proposed
to alleviate this issue, model bias still poses a great challenge for MBRL. In
this work, we propose to leverage the prior knowledge of underlying physics of
the environment, where the governing laws are (partially) known. In particular,
we developed a physics-informed MBRL framework, where governing equations and
physical constraints are utilized to inform the model learning and policy
search. By incorporating the prior information of the environment, the quality
of the learned model can be notably improved, while the required interactions
with the environment are significantly reduced, leading to better sample
efficiency and learning performance. The effectiveness and merit have been
demonstrated over a handful of classic control problems, where the environments
are governed by canonical ordinary/partial differential equations.
