Robust Convergence in Federated Learning through Label-wise Clustering
  Non-IID dataset and heterogeneous environment of the local clients are
regarded as a major issue in Federated Learning (FL), causing a downturn in the
convergence without achieving satisfactory performance. In this paper, we
propose a novel Label-wise clustering algorithm that guarantees the
trainability among geographically dispersed heterogeneous local clients, by
selecting only the local models trained with a dataset that approximates into
uniformly distributed class labels, which is likely to obtain faster
minimization of the loss and increment the accuracy among the FL network.
Through conducting experiments on the suggested six common non-IID scenarios,
we empirically show that the vanilla FL aggregation model is incapable of
gaining robust convergence generating biased pre-trained local models and
drifting the local weights to mislead the trainability in the worst case.
Moreover, we quantitatively estimate the expected performance of the local
models before training, which offers a global server to select the optimal
clients, saving additional computational costs. Ultimately, in order to gain
resolution of the non-convergence in such non-IID situations, we design
clustering algorithms based on local input class labels, accommodating the
diversity and assorting clients that could lead the overall system to attain
the swift convergence as global training continues. Our paper shows that
proposed Label-wise clustering demonstrates prompt and robust convergence
compared to other FL algorithms when local training datasets are non-IID or
coexist with IID through multiple experiments.
