MapGo: Model-Assisted Policy Optimization for Goal-Oriented Tasks
  In Goal-oriented Reinforcement learning, relabeling the raw goals in past
experience to provide agents with hindsight ability is a major solution to the
reward sparsity problem. In this paper, to enhance the diversity of relabeled
goals, we develop FGI (Foresight Goal Inference), a new relabeling strategy
that relabels the goals by looking into the future with a learned dynamics
model. Besides, to improve sample efficiency, we propose to use the dynamics
model to generate simulated trajectories for policy training. By integrating
these two improvements, we introduce the MapGo framework (Model-Assisted Policy
Optimization for Goal-oriented tasks). In our experiments, we first show the
effectiveness of the FGI strategy compared with the hindsight one, and then
show that the MapGo framework achieves higher sample efficiency when compared
to model-free baselines on a set of complicated tasks.
