Curriculum Design for Teaching via Demonstrations: Theory and
  Applications
  We consider the problem of teaching via demonstrations in sequential
decision-making settings. In particular, we study how to design a personalized
curriculum over demonstrations to speed up the learner's convergence. We
provide a unified curriculum strategy for two popular learner models: Maximum
Causal Entropy Inverse Reinforcement Learning (MaxEnt-IRL) and Cross-Entropy
Behavioral Cloning (CrossEnt-BC). Our unified strategy induces a ranking over
demonstrations based on a notion of difficulty scores computed w.r.t. the
teacher's optimal policy and the learner's current policy. Compared to the
state of the art, our strategy doesn't require access to the learner's internal
dynamics and still enjoys similar convergence guarantees under mild technical
conditions. Furthermore, we adapt our curriculum strategy to the setting where
no teacher agent is present using task-specific difficulty scores. Experiments
on a synthetic car driving environment and navigation-based environments
demonstrate the effectiveness of our curriculum strategy.
