Coded Consensus Monte Carlo: Robust One-Shot Distributed Bayesian
  Learning with Stragglers
  This letter studies distributed Bayesian learning in a setting encompassing a
central server and multiple workers by focusing on the problem of mitigating
the impact of stragglers. The standard one-shot, or embarrassingly parallel,
Bayesian learning protocol known as consensus Monte Carlo (CMC) is generalized
by proposing two straggler-resilient solutions based on grouping and coding.
The proposed methods, referred to as Group-based CMC (G-CMC) and Coded CMC
(C-CMC), leverage redundant computing at the workers in order to enable the
estimation of global posterior samples at the server based on partial outputs
from the workers. Simulation results show that C-CMC may outperform G-GCMC for
a small number of workers, while G-CMC is generally preferable for a larger
number of workers.
