OFedQIT: Communication-Efficient Online Federated Learning via Quantization and Intermittent Transmission
Online federated learning (OFL) is a promising framework to collaboratively
learn a sequence of non-linear functions (or models) from distributed streaming
data incoming to multiple clients while keeping the privacy of their local
data. In this framework, we first construct a vanilla method (named OFedAvg) by
incorporating online gradient descent (OGD) into the de facto aggregation
method (named FedAvg). Despite its optimal asymptotic performance, OFedAvg
suffers from heavy communication overhead and long learning delay. To tackle
these shortcomings, we propose a communication-efficient OFL algorithm (named
OFedQIT) by means of a stochastic quantization and an intermittent
transmission. Our major contribution is to theoretically prove that OFedQIT
over $T$ time slots can achieve an optimal sublinear regret bound
$\mathcal{O}(\sqrt{T})$ for any real data (including non-IID data) while
significantly reducing the communication overhead. Furthermore, this optimality
is still guaranteed even when a small fraction of clients (having faster
processing time and high-quality communication channel) in a network are
participated at once. Our analysis reveals that OFedQIT successfully addresses
the drawbacks of OFedAvg while maintaining superior learning accuracy.
Experiments with real datasets demonstrate the effectiveness of our algorithm
on various online classification and regression tasks.