Decentralized Bayesian Learning over Graphs
  We propose a decentralized learning algorithm over a general social network.
The algorithm leaves the training data distributed on the mobile devices while
utilizing a peer to peer model aggregation method. The proposed algorithm
allows agents with local data to learn a shared model explaining the global
training data in a decentralized fashion. The proposed algorithm can be viewed
as a Bayesian and peer-to-peer variant of federated learning in which each
agent keeps a "posterior probability distribution" over a global model
parameters. The agent update its "posterior" based on 1) the local training
data and 2) the asynchronous communication and model aggregation with their
1-hop neighbors. This Bayesian formulation allows for a systematic treatment of
model aggregation over any arbitrary connected graph. Furthermore, it provides
strong analytic guarantees on converge in the realizable case as well as a
closed form characterization of the rate of convergence. We also show that our
methodology can be combined with efficient Bayesian inference techniques to
train Bayesian neural networks in a decentralized manner. By empirical studies
we show that our theoretical analysis can guide the design of network/social
interactions and data partitioning to achieve convergence.
