Decentralized Federated Learning: A Segmented Gossip Approach
  The emerging concern about data privacy and security has motivated the
proposal of federated learning, which allows nodes to only synchronize the
locally-trained models instead their own original data. Conventional federated
learning architecture, inherited from the parameter server design, relies on
highly centralized topologies and the assumption of large nodes-to-server
bandwidths. However, in real-world federated learning scenarios the network
capacities between nodes are highly uniformly distributed and smaller than that
in a datacenter. It is of great challenges for conventional federated learning
approaches to efficiently utilize network capacities between nodes. In this
paper, we propose a model segment level decentralized federated learning to
tackle this problem. In particular, we propose a segmented gossip approach,
which not only makes full utilization of node-to-node bandwidth, but also has
good training convergence. The experimental results show that even the training
time can be highly reduced as compared to centralized federated learning.
