Learning Behavioral Soft Constraints from Demonstrations
  Many real-life scenarios require humans to make difficult trade-offs: do we
always follow all the traffic rules or do we violate the speed limit in an
emergency? These scenarios force us to evaluate the trade-off between
collective rules and norms with our own personal objectives and desires. To
create effective AI-human teams, we must equip AI agents with a model of how
humans make these trade-offs in complex environments when there are implicit
and explicit rules and constraints. Agent equipped with these models will be
able to mirror human behavior and/or to draw human attention to situations
where decision making could be improved. To this end, we propose a novel
inverse reinforcement learning (IRL) method: Max Entropy Inverse Soft
Constraint IRL (MESC-IRL), for learning implicit hard and soft constraints over
states, actions, and state features from demonstrations in deterministic and
non-deterministic environments modeled as Markov Decision Processes (MDPs). Our
method enables agents implicitly learn human constraints and desires without
the need for explicit modeling by the agent designer and to transfer these
constraints between environments. Our novel method generalizes prior work which
only considered deterministic hard constraints and achieves state of the art
performance.
