IR-VIC: Unsupervised Discovery of Sub-goals for Transfer in RL
  We propose a novel framework to identify sub-goals useful for exploration in
sequential decision making tasks under partial observability. We utilize the
variational intrinsic control framework (Gregor et.al., 2016) which maximizes
empowerment -- the ability to reliably reach a diverse set of states and show
how to identify sub-goals as states with high necessary option information
through an information theoretic regularizer. Despite being discovered without
explicit goal supervision, our sub-goals provide better exploration and sample
complexity on challenging grid-world navigation tasks compared to supervised
counterparts in prior work.
