Sample Efficient Reinforcement Learning via Model-Ensemble Exploration
  and Exploitation
  Model-based deep reinforcement learning has achieved success in various
domains that require high sample efficiencies, such as Go and robotics.
However, there are some remaining issues, such as planning efficient
explorations to learn more accurate dynamic models, evaluating the uncertainty
of the learned models, and more rational utilization of models. To mitigate
these issues, we present MEEE, a model-ensemble method that consists of
optimistic exploration and weighted exploitation. During exploration, unlike
prior methods directly selecting the optimal action that maximizes the expected
accumulative return, our agent first generates a set of action candidates and
then seeks out the optimal action that takes both expected return and future
observation novelty into account. During exploitation, different discounted
weights are assigned to imagined transition tuples according to their model
uncertainty respectively, which will prevent model predictive error propagation
in agent training. Experiments on several challenging continuous control
benchmark tasks demonstrated that our approach outperforms other model-free and
model-based state-of-the-art methods, especially in sample complexity.
