Training Speech Recognition Models with Federated Learning: A
  Quality/Cost Framework
  We propose using federated learning, a decentralized on-device learning
paradigm, to train speech recognition models. By performing epochs of training
on a per-user basis, federated learning must incur the cost of dealing with
non-IID data distributions, which are expected to negatively affect the quality
of the trained model. We propose a framework by which the degree of
non-IID-ness can be varied, consequently illustrating a trade-off between model
quality and the computational cost of federated training, which we capture
through a novel metric. Finally, we demonstrate that hyper-parameter
optimization and appropriate use of variational noise are sufficient to
compensate for the quality impact of non-IID distributions, while decreasing
the cost.
