Fast-Convergent Federated Learning with Adaptive Weighting
  Federated learning (FL) enables resource-constrained edge nodes to
collaboratively learn a global model under the orchestration of a central
server while keeping privacy-sensitive data locally. The
non-independent-and-identically-distributed (non-IID) data samples across
participating nodes slow model training and impose additional communication
rounds for FL to converge. In this paper, we propose Federated Adaptive
Weighting (FedAdp) algorithm that aims to accelerate model convergence under
the presence of nodes with non-IID dataset. We observe the implicit connection
between the node contribution to the global model aggregation and data
distribution on the local node through theoretical and empirical analysis. We
then propose to assign different weights for updating the global model based on
node contribution adaptively through each training round. The contribution of
participating nodes is first measured by the angle between the local gradient
vector and the global gradient vector, and then, weight is quantified by a
designed non-linear mapping function subsequently. The simple yet effective
strategy can reinforce positive (suppress negative) node contribution
dynamically, resulting in communication round reduction drastically. Its
superiority over the commonly adopted Federated Averaging (FedAvg) is verified
both theoretically and experimentally. With extensive experiments performed in
Pytorch and PySyft, we show that FL training with FedAdp can reduce the number
of communication rounds by up to 54.1% on MNIST dataset and up to 45.4% on
FashionMNIST dataset, as compared to FedAvg algorithm.
