A Contraction Approach to Model-based Reinforcement Learning
  Despite its experimental success, Model-based Reinforcement Learning still
lacks a complete theoretical understanding. To this end, we analyze the error
in the cumulative reward using a contraction approach. We consider both
stochastic and deterministic state transitions for continuous (non-discrete)
state and action spaces. This approach doesn't require strong assumptions and
can recover the typical quadratic error to the horizon. We prove that branched
rollouts can reduce this error and are essential for deterministic transitions
to have a Bellman contraction. Our analysis of policy mismatch error also
applies to Imitation Learning. In this case, we show that GAN-type learning has
an advantage over Behavioral Cloning when its discriminator is well-trained.
