Learner-aware Teaching: Inverse Reinforcement Learning with Preferences
  and Constraints
  Inverse reinforcement learning (IRL) enables an agent to learn complex
behavior by observing demonstrations from a (near-)optimal policy. The typical
assumption is that the learner's goal is to match the teacher's demonstrated
behavior. In this paper, we consider the setting where the learner has its own
preferences that it additionally takes into consideration. These preferences
can for example capture behavioral biases, mismatched worldviews, or physical
constraints. We study two teaching approaches: learner-agnostic teaching, where
the teacher provides demonstrations from an optimal policy ignoring the
learner's preferences, and learner-aware teaching, where the teacher accounts
for the learner's preferences. We design learner-aware teaching algorithms and
show that significant performance improvements can be achieved over
learner-agnostic teaching.
