Production federated keyword spotting via distillation, filtering, and
  joint federated-centralized training
  We trained a keyword spotting model using federated learning on real user
devices and observed significant improvements when the model was deployed for
inference on phones. To compensate for data domains that are missing from
on-device training caches, we employed joint federated-centralized training.
And to learn in the absence of curated labels on-device, we formulated a
confidence filtering strategy based on user-feedback signals for federated
distillation. These techniques created models that significantly improved
quality metrics in offline evaluations and user-experience metrics in live A/B
experiments.
