AvE: Assistance via Empowerment
  One difficulty in using artificial agents for human-assistive applications
lies in the challenge of accurately assisting with a person's goal(s). Existing
methods tend to rely on inferring the human's goal, which is challenging when
there are many potential goals or when the set of candidate goals is difficult
to identify. We propose a new paradigm for assistance by instead increasing the
human's ability to control their environment, and formalize this approach by
augmenting reinforcement learning with human empowerment. This task-agnostic
objective preserves the person's autonomy and ability to achieve any eventual
state. We test our approach against assistance based on goal inference,
highlighting scenarios where our method overcomes failure modes stemming from
goal ambiguity or misspecification. As existing methods for estimating
empowerment in continuous domains are computationally hard, precluding its use
in real time learned assistance, we also propose an efficient
empowerment-inspired proxy metric. Using this, we are able to successfully
demonstrate our method in a shared autonomy user study for a challenging
simulated teleoperation task with human-in-the-loop training.
