Federated Multi-Mini-Batch: An Efficient Training Approach to Federated
  Learning in Non-IID Environments
  Federated learning has faced performance and network communication
challenges, especially in the environments where the data is not independent
and identically distributed (IID) across the clients. To address the former
challenge, we introduce the federated-centralized concordance property and show
that the federated single-mini-batch training approach can achieve comparable
performance as the corresponding centralized training in the Non-IID
environments. To deal with the latter, we present the federated
multi-mini-batch approach and illustrate that it can establish a trade-off
between the performance and communication efficiency and outperforms federated
averaging in the Non-IID settings.
