One-shot Federated Learning without Server-side Training
  Federated Learning (FL) has recently made significant progress as a new
machine learning paradigm for privacy protection. Due to the high communication
cost of traditional FL, one-shot federated learning is gaining popularity as a
way to reduce communication cost between clients and the server. Most of the
existing one-shot FL methods are based on Knowledge Distillation; however,
distillation based approach requires an extra training phase and depends on
publicly available data sets. In this work, we consider a novel and challenging
setting: performing a single round of parameter aggregation on the local models
without server-side training on a public data set. In this new setting, we
propose an effective algorithm for Model Aggregation via Exploring Common
Harmonized Optima (MA-Echo), which iteratively updates the parameters of all
local models to bring them close to a common low-loss area on the loss surface,
without harming performance on their own data sets at the same time. Compared
to the existing methods, MA-Echo can work well even in extremely non-identical
data distribution settings where the support categories of each local model
have no overlapped labels with those of the others. We conduct extensive
experiments on two popular image classification data sets to compare the
proposed method with existing methods and demonstrate the effectiveness of
MA-Echo, which clearly outperforms the state-of-the-arts.
