Multitask Adaptation by Retrospective Exploration with Learned World
  Models
  Model-based reinforcement learning (MBRL) allows solving complex tasks in a
sample-efficient manner. However, no information is reused between the tasks.
In this work, we propose a meta-learned addressing model called RAMa that
provides training samples for the MBRL agent taken from continuously growing
task-agnostic storage. The model is trained to maximize the expected agent's
performance by selecting promising trajectories solving prior tasks from the
storage. We show that such retrospective exploration can accelerate the
learning process of the MBRL agent by better informing learned dynamics and
prompting agent with exploratory trajectories. We test the performance of our
approach on several domains from the DeepMind control suite, from Metaworld
multitask benchmark, and from our bespoke environment implemented with a
robotic NVIDIA Isaac simulator to test the ability of the model to act in a
photorealistic, ray-traced environment.
