Federated Multi-Agent Actor-Critic Learning for Age Sensitive Mobile
  Edge Computing
  As an emerging technique, mobile edge computing (MEC) introduces a new
processing scheme for various distributed communication-computing systems such
as industrial Internet of Things (IoT), vehicular communication, smart city,
etc. In this work, we mainly focus on the timeliness of the MEC systems where
the freshness of the data and computation tasks is significant. Firstly, we
formulate a kind of age-sensitive MEC models and define the average age of
information (AoI) minimization problems of interests. Then, a novel policy
based multi-agent deep reinforcement learning (RL) framework, called
heterogeneous multi-agent actor critic (H-MAAC), is proposed as a paradigm for
joint collaboration in the investigated MEC systems, where edge devices and
center controller learn the interactive strategies through their own
observations. To improves the system performance, we develop the corresponding
online algorithm by introducing an edge federated learning mode into the
multi-agent cooperation whose advantages on learning convergence can be
guaranteed theoretically. To the best of our knowledge, it's the first joint
MEC collaboration algorithm that combines the edge federated mode with the
multi-agent actor-critic reinforcement learning. Furthermore, we evaluate the
proposed approach and compare it with classical RL based methods. As a result,
the proposed framework not only outperforms the baseline on average system age,
but also promotes the stability of training process. Besides, the simulation
results provide some innovative perspectives for the system design under the
edge federated collaboration.
