FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling
  and Correction
  Federated learning (FL) allows multiple clients to collectively train a
high-performance global model without sharing their private data. However, the
key challenge in federated learning is that the clients have significant
statistical heterogeneity among their local data distributions, which would
cause inconsistent optimized local models on the client-side. To address this
fundamental dilemma, we propose a novel federated learning algorithm with local
drift decoupling and correction (FedDC). Our FedDC only introduces lightweight
modifications in the local training phase, in which each client utilizes an
auxiliary local drift variable to track the gap between the local model
parameter and the global model parameters. The key idea of FedDC is to utilize
this learned local drift variable to bridge the gap, i.e., conducting
consistency in parameter-level. The experiment results and analysis demonstrate
that FedDC yields expediting convergence and better performance on various
image classification tasks, robust in partial participation settings, non-iid
data, and heterogeneous clients.
