A Practical Data-Free Approach to One-shot Federated Learning with
  Heterogeneity
  One-shot Federated Learning (FL) has recently emerged as a promising
approach, which allows the central server to learn a model in a single
communication round. Despite the low communication cost, existing one-shot FL
methods are mostly impractical or face inherent limitations, e.g., a public
dataset is required, clients' models are homogeneous, need to upload additional
data/model information. To overcome these issues, we propose a more practical
data-free approach named FedSyn for one-shot FL framework with heterogeneity.
Our FedSyn trains the global model by a data generation stage and a model
distillation stage. To the best of our knowledge, FedSyn is the first method
that can be practically applied to various real-world applications due to the
following advantages: (1) FedSyn requires no additional information (except the
model parameters) to be transferred between clients and the server; (2) FedSyn
does not require any auxiliary dataset for training; (3) FedSyn is the first to
consider both model and statistical heterogeneities in FL, i.e., the clients'
data are non-iid and different clients may have different model architectures.
Experiments on a variety of real-world datasets demonstrate the superiority of
our FedSyn. For example, FedSyn outperforms the best baseline method Fed-ADI by
5.08% on CIFAR10 dataset when data are non-iid.
