Hierarchical Soft Actor-Critic: Adversarial Exploration via Mutual
  Information Optimization
  We describe a novel extension of soft actor-critics for hierarchical Deep
Q-Networks (HDQN) architectures using mutual information metric. The proposed
extension provides a suitable framework for encouraging explorations in such
hierarchical networks. A natural utilization of this framework is an
adversarial setting, where meta-controller and controller play minimax over the
mutual information objective but cooperate on maximizing expected rewards.
