WFA-IRL: Inverse Reinforcement Learning of Autonomous Behaviors Encoded
  as Weighted Finite Automata
  This paper presents a method for learning logical task specifications and
cost functions from demonstrations. Linear temporal logic (LTL) formulas are
widely used to express complex objectives and constraints for autonomous
systems. Yet, such specifications may be challenging to construct by hand.
Instead, we consider demonstrated task executions, whose temporal logic
structure and transition costs need to be inferred by an autonomous agent. We
employ a spectral learning approach to extract a weighted finite automaton
(WFA), approximating the unknown logic structure of the task. Thereafter, we
define a product between the WFA for high-level task guidance and a labeled
Markov decision process for low-level control. An inverse reinforcement
learning (IRL) problem is considered to learn a cost function by
backpropagating the loss between agent and expert behaviors through the
planning algorithm. Our proposed model, termed WFA-IRL, is capable of
generalizing the execution of the inferred task specification in a suite of
MiniGrid environments.
