Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented
  Demonstrations using Directed Information
  The use of imitation learning to learn a single policy for a complex task
that has multiple modes or hierarchical structure can be challenging. In fact,
previous work has shown that when the modes are known, learning separate
policies for each mode or sub-task can greatly improve the performance of
imitation learning. In this work, we discover the interaction between sub-tasks
from their resulting state-action trajectory sequences using a directed
graphical model. We propose a new algorithm based on the generative adversarial
imitation learning framework which automatically learns sub-task policies from
unsegmented demonstrations. Our approach maximizes the directed information
flow in the graphical model between sub-task latent variables and their
generated trajectories. We also show how our approach connects with the
existing Options framework, which is commonly used to learn hierarchical
policies.
