DACFL: Dynamic Average Consensus Based Federated Learning in
  Decentralized Topology
  Federated learning (FL) is a burgeoning distributed machine learning
framework where a central parameter server (PS) coordinates many local users to
train a globally consistent model. Conventional federated learning inevitably
relies on a centralized topology with a PS. As a result, it will paralyze once
the PS fails. To alleviate such a single point failure, especially on the PS,
some existing work has provided decentralized FL (DFL) implementations like
CDSGD and D-PSGD to facilitate FL in a decentralized topology. However, there
are still some problems with these methods, e.g., significant divergence
between users' final models in CDSGD and a network-wide model average necessity
in D-PSGD. In order to solve these deficiency, this paper devises a new DFL
implementation coined as DACFL, where each user trains its model using its own
training data and exchanges the intermediate models with its neighbors through
a symmetric and doubly stochastic matrix. The DACFL treats the progress of each
user's local training as a discrete-time process and employs a first order
dynamic average consensus (FODAC) method to track the \textit{average model} in
the absence of the PS. In this paper, we also provide a theoretical convergence
analysis of DACFL on the premise of i.i.d data to strengthen its rationality.
The experimental results on MNIST, Fashion-MNIST and CIFAR-10 validate the
feasibility of our solution in both time-invariant and time-varying network
topologies, and declare that DACFL outperforms D-PSGD and CDSGD in most cases.
