Generalized Inverse Planning: Learning Lifted non-Markovian Utility for
  Generalizable Task Representation
  In searching for a generalizable representation of temporally extended tasks,
we spot two necessary constituents: the utility needs to be non-Markovian to
transfer temporal relations invariant to a probability shift, the utility also
needs to be lifted to abstract out specific grounding objects. In this work, we
study learning such utility from human demonstrations. While inverse
reinforcement learning (IRL) has been accepted as a general framework of
utility learning, its fundamental formulation is one concrete Markov Decision
Process. Thus the learned reward function does not specify the task
independently of the environment. Going beyond that, we define a domain of
generalization that spans a set of planning problems following a schema. We
hence propose a new quest, Generalized Inverse Planning, for utility learning
in this domain. We further outline a computational framework, Maximum Entropy
Inverse Planning (MEIP), that learns non-Markovian utility and associated
concepts in a generative manner. The learned utility and concepts form a task
representation that generalizes regardless of probability shift or structural
change. Seeing that the proposed generalization problem has not been widely
studied yet, we carefully define an evaluation protocol, with which we
illustrate the effectiveness of MEIP on two proof-of-concept domains and one
challenging task: learning to fold from demonstrations.
