Data Driven Control with Learned Dynamics: Model-Based versus Model-Free
  Approach
  This paper compares two different types of data-driven control methods,
representing model-based and model-free approaches. One is a recently proposed
method - Deep Koopman Representation for Control (DKRC), which utilizes a deep
neural network to map an unknown nonlinear dynamical system to a
high-dimensional linear system, which allows for employing state-of-the-art
control strategy. The other one is a classic model-free control method based on
an actor-critic architecture - Deep Deterministic Policy Gradient (DDPG), which
has been proved to be effective in various dynamical systems. The comparison is
carried out in OpenAI Gym, which provides multiple control environments for
benchmark purposes. Two examples are provided for comparison, i.e., classic
Inverted Pendulum and Lunar Lander Continuous Control. From the results of the
experiments, we compare these two methods in terms of control strategies and
the effectiveness under various initialization conditions. We also examine the
learned dynamic model from DKRC with the analytical model derived from the
Euler-Lagrange Linearization method, which demonstrates the accuracy in the
learned model for unknown dynamics from a data-driven sample-efficient
approach.
