Energy-Based Continuous Inverse Optimal Control
  The problem of continuous inverse optimal control (over finite time horizon)
is to learn the unknown cost function over the sequence of continuous control
variables from expert demonstrations. In this article, we study this
fundamental problem in the framework of energy-based model, where the observed
expert trajectories are assumed to be random samples from a probability density
function defined as the exponential of the negative cost function up to a
normalizing constant. The parameters of the cost function are learned by
maximum likelihood via an "analysis by synthesis" scheme, which iterates (1)
synthesis step: sample the synthesized trajectories from the current
probability density using the Langevin dynamics via back-propagation through
time, and (2) analysis step: update the model parameters based on the
statistical difference between the synthesized trajectories and the observed
trajectories. Given the fact that an efficient optimization algorithm is
usually available for an optimal control problem, we also consider a convenient
approximation of the above learning method, where we replace the sampling in
the synthesis step by optimization. Moreover, to make the sampling or
optimization more efficient, we propose to train the energy-based model
simultaneously with a top-down trajectory generator via cooperative learning,
where the trajectory generator is used to fast initialize the synthesis step of
the energy-based model. We demonstrate the proposed methods on autonomous
driving tasks, and show that they can learn suitable cost functions for optimal
control.
