A Bayesian Federated Learning Framework with Online Laplace
  Approximation
  Federated learning (FL) allows multiple clients to collaboratively learn a
globally shared model through cycles of model aggregation and local model
training, without the need to share data. Most existing FL methods train local
models separately on different clients, and then simply average their
parameters to obtain a centralized model on the server side. However, these
approaches generally suffer from large aggregation errors and severe local
forgetting, which are particularly bad in heterogeneous data settings. To
tackle these issues, in this paper, we propose a novel FL framework that uses
online Laplace approximation to approximate posteriors on both the client and
server side. On the server side, a multivariate Gaussian product mechanism is
employed to construct and maximize a global posterior, largely reducing the
aggregation errors induced by large discrepancies between local models. On the
client side, a prior loss that uses the global posterior probabilistic
parameters delivered from the server is designed to guide the local training.
Binding such learning constraints from other clients enables our method to
mitigate local forgetting. Finally, we achieve state-of-the-art results on
several benchmarks, clearly demonstrating the advantages of the proposed
method.
