Provable Hierarchical Lifelong Learning with a Sketch-based Modular
  Architecture
  We propose a modular architecture for the lifelong learning of hierarchically
structured tasks. Specifically, we prove that our architecture is theoretically
able to learn tasks that can be solved by functions that are learnable given
access to functions for other, previously learned tasks as subroutines. We
empirically show that some tasks that we can learn in this way are not learned
by standard training methods in practice; indeed, prior work suggests that some
such tasks cannot be learned by any efficient method without the aid of the
simpler tasks. We also consider methods for identifying the tasks
automatically, without relying on explicitly given indicators.
