HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation
  with Mobile Manipulators
  Most common navigation tasks in human environments require auxiliary arm
interactions, e.g. opening doors, pressing buttons and pushing obstacles away.
This type of navigation tasks, which we call Interactive Navigation, requires
the use of mobile manipulators: mobile bases with manipulation capabilities.
Interactive Navigation tasks are usually long-horizon and composed of
heterogeneous phases of pure navigation, pure manipulation, and their
combination. Using the wrong part of the embodiment is inefficient and hinders
progress. We propose HRL4IN, a novel Hierarchical RL architecture for
Interactive Navigation tasks. HRL4IN exploits the exploration benefits of HRL
over flat RL for long-horizon tasks thanks to temporally extended commitments
towards subgoals. Different from other HRL solutions, HRL4IN handles the
heterogeneous nature of the Interactive Navigation task by creating subgoals in
different spaces in different phases of the task. Moreover, HRL4IN selects
different parts of the embodiment to use for each phase, improving energy
efficiency. We evaluate HRL4IN against flat PPO and HAC, a state-of-the-art HRL
algorithm, on Interactive Navigation in two environments - a 2D grid-world
environment and a 3D environment with physics simulation. We show that HRL4IN
significantly outperforms its baselines in terms of task performance and energy
efficiency. More information is available at
https://sites.google.com/view/hrl4in.
