Asynchronous Online Federated Learning for Edge Devices with Non-IID
  Data
  Federated learning (FL) is a machine learning paradigm where a shared central
model is learned across distributed edge devices while the training data
remains on these devices. Federated Averaging (FedAvg) is the leading
optimization method for training non-convex models in this setting with a
synchronized protocol. However, the assumptions made by FedAvg are not
realistic given the heterogeneity of devices. In particular, the volume and
distribution of collected data vary in the training process due to different
sampling rates of edge devices. The edge devices themselves also vary in their
available communication bandwidth and system configurations, such as memory,
processor speed, and power requirements. This leads to vastly different
training times as well as model/data transfer times. Furthermore, availability
issues at edge devices can lead to a lack of contribution from specific edge
devices to the federated model. In this paper, we present an Asynchronous
Online Federated Learning (ASO-Fed) framework, where the edge devices perform
online learning with continuous streaming local data and a central server
aggregates model parameters from clients. Our framework updates the central
model in an asynchronous manner to tackle the challenges associated with both
varying computational loads at heterogeneous edge devices and edge devices that
lag behind or dropout. We perform extensive experiments on a simulated
benchmark image dataset and three real-world non-IID streaming datasets. The
results demonstrate the effectiveness of \model~on converging fast and
maintaining good prediction performance.
