Weight Divergence Driven Divide-and-Conquer Approach for Optimal
  Federated Learning from non-IID Data
  Federated Learning allows training of data stored in distributed devices
without the need for centralizing training data, thereby maintaining data
privacy. Addressing the ability to handle data heterogeneity (non-identical and
independent distribution or non-IID) is a key enabler for the wider deployment
of Federated Learning. In this paper, we propose a novel Divide-and-Conquer
training methodology that enables the use of the popular FedAvg aggregation
algorithm by overcoming the acknowledged FedAvg limitations in non-IID
environments. We propose a novel use of Cosine-distance based Weight Divergence
metric to determine the exact point where a Deep Learning network can be
divided into class agnostic initial layers and class-specific deep layers for
performing a Divide and Conquer training. We show that the methodology achieves
trained model accuracy at par (and in certain cases exceeding) with numbers
achieved by state-of-the-art Aggregation algorithms like FedProx, FedMA, etc.
Also, we show that this methodology leads to compute and bandwidth
optimizations under certain documented conditions.
