Fractional Transfer Learning for Deep Model-Based Reinforcement Learning
  Reinforcement learning (RL) is well known for requiring large amounts of data
in order for RL agents to learn to perform complex tasks. Recent progress in
model-based RL allows agents to be much more data-efficient, as it enables them
to learn behaviors of visual environments in imagination by leveraging an
internal World Model of the environment. Improved sample efficiency can also be
achieved by reusing knowledge from previously learned tasks, but transfer
learning is still a challenging topic in RL. Parameter-based transfer learning
is generally done using an all-or-nothing approach, where the network's
parameters are either fully transferred or randomly initialized. In this work
we present a simple alternative approach: fractional transfer learning. The
idea is to transfer fractions of knowledge, opposed to discarding potentially
useful knowledge as is commonly done with random initialization. Using the
World Model-based Dreamer algorithm, we identify which type of components this
approach is applicable to, and perform experiments in a new multi-source
transfer learning setting. The results show that fractional transfer learning
often leads to substantially improved performance and faster learning compared
to learning from scratch and random initialization.
