Comparative assessment of federated and centralized machine learning
  Federated Learning (FL) is a privacy preserving machine learning scheme,
where training happens with data federated across devices and not leaving them
to sustain user privacy. This is ensured by making the untrained or partially
trained models to reach directly the individual devices and getting locally
trained "on-device" using the device owned data, and the server aggregating all
the partially trained model learnings to update a global model. Although almost
all the model learning schemes in the federated learning setup use gradient
descent, there are certain characteristic differences brought about by the
non-IID nature of the data availability, that affects the training in
comparison to the centralized schemes. In this paper, we discuss the various
factors that affect the federated learning training, because of the non-IID
distributed nature of the data, as well as the inherent differences in the
federating learning approach as against the typical centralized gradient
descent techniques. We empirically demonstrate the effect of number of samples
per device and the distribution of output labels on federated learning. In
addition to the privacy advantage we seek through federated learning, we also
study if there is a cost advantage while using federated learning frameworks.
We show that federated learning does have an advantage in cost when the model
sizes to be trained are not reasonably large. All in all, we present the need
for careful design of model for both performance and cost.
