Stochastic Optimal Control as Approximate Input Inference
  Optimal control of stochastic nonlinear dynamical systems is a major
challenge in the domain of robot learning. Given the intractability of the
global control problem, state-of-the-art algorithms focus on approximate
sequential optimization techniques, that heavily rely on heuristics for
regularization in order to achieve stable convergence. By building upon the
duality between inference and control, we develop the view of Optimal Control
as Input Estimation, devising a probabilistic stochastic optimal control
formulation that iteratively infers the optimal input distributions by
minimizing an upper bound of the control cost. Inference is performed through
Expectation Maximization and message passing on a probabilistic graphical model
of the dynamical system, and time-varying linear Gaussian feedback controllers
are extracted from the joint state-action distribution. This perspective
incorporates uncertainty quantification, effective initialization through
priors, and the principled regularization inherent to the Bayesian treatment.
Moreover, it can be shown that for deterministic linearized systems, our
framework derives the maximum entropy linear quadratic optimal control law. We
provide a complete and detailed derivation of our probabilistic approach and
highlight its advantages in comparison to other deterministic and probabilistic
solvers.
