Event-Based Communication in Distributed Q-Learning
  We present an approach to reduce the communication of information needed on a
Distributed Q-Learning system inspired by Event Triggered Control (ETC)
techniques. We consider a baseline scenario of a distributed Q-learning problem
on a Markov Decision Process (MDP). Following an event-based approach, N agents
explore the MDP and communicate experiences to a central learner only when
necessary, which performs updates of the actor Q functions. We design an Event
Based distributed Q learning system (EBd-Q), and derive convergence guarantees
with respect to a vanilla Q-learning algorithm. We present experimental results
showing that event-based communication results in a substantial reduction of
data transmission rates in such distributed systems. Additionally, we discuss
what effects (desired and undesired) these event-based approaches have on the
learning processes studied, and how they can be applied to more complex
multi-agent systems.
