Time-Correlated Sparsification for Efficient Over-the-Air Model
  Aggregation in Wireless Federated Learning
  Federated edge learning (FEEL) is a promising distributed machine learning
(ML) framework to drive edge intelligence applications. However, due to the
dynamic wireless environments and the resource limitations of edge devices,
communication becomes a major bottleneck. In this work, we propose
time-correlated sparsification with hybrid aggregation (TCS-H) for
communication-efficient FEEL, which exploits jointly the power of model
compression and over-the-air computation. By exploiting the temporal
correlations among model parameters, we construct a global sparsification mask,
which is identical across devices, and thus enables efficient model aggregation
over-the-air. Each device further constructs a local sparse vector to explore
its own important parameters, which are aggregated via digital communication
with orthogonal multiple access. We further design device scheduling and power
allocation algorithms for TCS-H. Experiment results show that, under limited
communication resources, TCS-H can achieve significantly higher accuracy
compared to the conventional top-K sparsification with orthogonal model
aggregation, with both i.i.d. and non-i.i.d. data distributions.
