Modular Adaptive Policy Selection for Multi-Task Imitation Learning
  through Task Division
  Deep imitation learning requires many expert demonstrations, which can be
hard to obtain, especially when many tasks are involved. However, different
tasks often share similarities, so learning them jointly can greatly benefit
them and alleviate the need for many demonstrations. But, joint multi-task
learning often suffers from negative transfer, sharing information that should
be task-specific. In this work, we introduce a method to perform multi-task
imitation while allowing for task-specific features. This is done by using
proto-policies as modules to divide the tasks into simple sub-behaviours that
can be shared. The proto-policies operate in parallel and are adaptively chosen
by a selector mechanism that is jointly trained with the modules. Experiments
on different sets of tasks show that our method improves upon the accuracy of
single agents, task-conditioned and multi-headed multi-task agents, as well as
state-of-the-art meta learning agents. We also demonstrate its ability to
autonomously divide the tasks into both shared and task-specific
sub-behaviours.
