Dropout's Dream Land: Generalization from Learned Simulators to Reality
  A World Model is a generative model used to simulate an environment. World
Models have proven capable of learning spatial and temporal representations of
Reinforcement Learning environments. In some cases, a World Model offers an
agent the opportunity to learn entirely inside of its own dream environment. In
this work we explore improving the generalization capabilities from dream
environments to real environments (Dream2Real). We present a general approach
to improve a controller's ability to transfer from a neural network dream
environment to reality at little additional cost. These improvements are gained
by drawing on inspiration from Domain Randomization, where the basic idea is to
randomize as much of a simulator as possible without fundamentally changing the
task at hand. Generally, Domain Randomization assumes access to a pre-built
simulator with configurable parameters but oftentimes this is not available. By
training the World Model using dropout, the dream environment is capable of
creating a nearly infinite number of different dream environments. Previous use
cases of dropout either do not use dropout at inference time or averages the
predictions generated by multiple sampled masks (Monte-Carlo Dropout).
Dropout's Dream Land leverages each unique mask to create a diverse set of
dream environments. Our experimental results show that Dropout's Dream Land is
an effective technique to bridge the reality gap between dream environments and
reality. Furthermore, we additionally perform an extensive set of ablation
studies.
