FedNNNN: Norm-Normalized Neural Network Aggregation for Fast and
  Accurate Federated Learning
  Federated learning (FL) is a distributed learning protocol in which a server
needs to aggregate a set of models learned some independent clients to proceed
the learning process. At present, model averaging, known as FedAvg, is one of
the most widely adapted aggregation techniques. However, it is known to yield
the models with degraded prediction accuracy and slow convergence. In this
work, we find out that averaging models from different clients significantly
diminishes the norm of the update vectors, resulting in slow learning rate and
low prediction accuracy. Therefore, we propose a new aggregation method called
FedNNNN. Instead of simple model averaging, we adjust the norm of the update
vector and introduce momentum control techniques to improve the aggregation
effectiveness of FL. As a demonstration, we evaluate FedNNNN on multiple
datasets and scenarios with different neural network models, and observe up to
5.4% accuracy improvement.
