Federated Expectation Maximization with heterogeneity mitigation and
  variance reduction
  The Expectation Maximization (EM) algorithm is the default algorithm for
inference in latent variable models. As in any other field of machine learning,
applications of latent variable models to very large datasets make the use of
advanced parallel and distributed architectures mandatory. This paper
introduces FedEM, which is the first extension of the EM algorithm to the
federated learning context. FedEM is a new communication efficient method,
which handles partial participation of local devices, and is robust to
heterogeneous distributions of the datasets. To alleviate the communication
bottleneck, FedEM compresses appropriately defined complete data sufficient
statistics. We also develop and analyze an extension of FedEM to further
incorporate a variance reduction scheme. In all cases, we derive finite-time
complexity bounds for smooth non-convex problems. Numerical results are
presented to support our theoretical findings, as well as an application to
federated missing values imputation for biodiversity monitoring.
