Analysis and Optimal Edge Assignment For Hierarchical Federated Learning
  on Non-IID Data
  Distributed learning algorithms aim to leverage distributed and diverse data
stored at users' devices to learn a global phenomena by performing training
amongst participating devices and periodically aggregating their local models'
parameters into a global model. Federated learning is a promising paradigm that
allows for extending local training among the participant devices before
aggregating the parameters, offering better communication efficiency. However,
in the cases where the participants' data are strongly skewed (i.e., non-IID),
the local models can overfit local data, leading to low performing global
model. In this paper, we first show that a major cause of the performance drop
is the weighted distance between the distribution over classes on users'
devices and the global distribution. Then, to face this challenge, we leverage
the edge computing paradigm to design a hierarchical learning system that
performs Federated Gradient Descent on the user-edge layer and Federated
Averaging on the edge-cloud layer. In this hierarchical architecture, we
formalize and optimize this user-edge assignment problem such that edge-level
data distributions turn to be similar (i.e., close to IID), which enhances the
Federated Averaging performance. Our experiments on multiple real-world
datasets show that the proposed optimized assignment is tractable and leads to
faster convergence of models towards a better accuracy value.
