Decentralized Federated Learning: Balancing Communication and Computing
  Costs
  Decentralized stochastic gradient descent (SGD) is a driving engine for
decentralized federated learning (DFL). The performance of decentralized SGD is
jointly influenced by inter-node communications and local updates. In this
paper, we propose a general DFL framework, which implements both multiple local
updates and multiple inter-node communications periodically, to strike a
balance between communication efficiency and model consensus. It can provide a
general decentralized SGD analytical framework. We establish strong convergence
guarantees for the proposed DFL algorithm without the assumption of convex
objectives. The convergence rate of DFL can be optimized to achieve the balance
of communication and computing costs under constrained resources. For improving
communication efficiency of DFL, compressed communication is further introduced
to the proposed DFL as a new scheme, named DFL with compressed communication
(C-DFL). The proposed C-DFL exhibits linear convergence for strongly convex
objectives. Experiment results based on MNIST and CIFAR-10 datasets illustrate
the superiority of DFL over traditional decentralized SGD methods and show that
C-DFL further enhances communication efficiency.
