Discovering hierarchies using Imitation Learning from hierarchy aware
  policies
  Learning options that allow agents to exhibit temporally higher order
behavior has proven to be useful in increasing exploration, reducing sample
complexity and for various transfer scenarios. Deep Discovery of Options (DDO)
is a generative algorithm that learns a hierarchical policy along with options
directly from expert trajectories. We perform a qualitative and quantitative
analysis of options inferred from DDO in different domains. To this end, we
suggest different value metrics like option termination condition, hinge value
function error and KL-Divergence based distance metric to compare different
methods. Analyzing the termination condition of the options and number of time
steps the options were run revealed that the options were terminating
prematurely. We suggest modifications which can be incorporated easily and
alleviates the problem of shorter options and a collapse of options to the same
mode.
