Decentralized federated learning of deep neural networks on non-iid data
  We tackle the non-convex problem of learning a personalized deep learning
model in a decentralized setting. More specifically, we study decentralized
federated learning, a peer-to-peer setting where data is distributed among many
clients and where there is no central server to orchestrate the training. In
real world scenarios, the data distributions are often heterogeneous between
clients. Therefore, in this work we study the problem of how to efficiently
learn a model in a peer-to-peer system with non-iid client data. We propose a
method named Performance-Based Neighbor Selection (PENS) where clients with
similar data distributions detect each other and cooperate by evaluating their
training losses on each other's data to learn a model suitable for the local
data distribution. Our experiments on benchmark datasets show that our proposed
method is able to achieve higher accuracies as compared to strong baselines.
