Inverse Reinforcement Learning with Simultaneous Estimation of Rewards
  and Dynamics
  Inverse Reinforcement Learning (IRL) describes the problem of learning an
unknown reward function of a Markov Decision Process (MDP) from observed
behavior of an agent. Since the agent's behavior originates in its policy and
MDP policies depend on both the stochastic system dynamics as well as the
reward function, the solution of the inverse problem is significantly
influenced by both. Current IRL approaches assume that if the transition model
is unknown, additional samples from the system's dynamics are accessible, or
the observed behavior provides enough samples of the system's dynamics to solve
the inverse problem accurately. These assumptions are often not satisfied. To
overcome this, we present a gradient-based IRL approach that simultaneously
estimates the system's dynamics. By solving the combined optimization problem,
our approach takes into account the bias of the demonstrations, which stems
from the generating policy. The evaluation on a synthetic MDP and a transfer
learning task shows improvements regarding the sample efficiency as well as the
accuracy of the estimated reward functions and transition models.
