Multi-Agent Reinforcement Learning with Temporal Logic Specifications
  In this paper, we study the problem of learning to satisfy temporal logic
specifications with a group of agents in an unknown environment, which may
exhibit probabilistic behaviour. From a learning perspective these
specifications provide a rich formal language with which to capture tasks or
objectives, while from a logic and automated verification perspective the
introduction of learning capabilities allows for practical applications in
large, stochastic, unknown environments. The existing work in this area is,
however, limited. Of the frameworks that consider full linear temporal logic or
have correctness guarantees, all methods thus far consider only the case of a
single temporal logic specification and a single agent. In order to overcome
this limitation, we develop the first multi-agent reinforcement learning
technique for temporal logic specifications, which is also novel in its ability
to handle multiple specifications. We provide correctness and convergence
guarantees for our main algorithm - ALMANAC (Automaton/Logic Multi-Agent
Natural Actor-Critic) - even when using function approximation. Alongside our
theoretical results, we further demonstrate the applicability of our technique
via a set of preliminary experiments.
