SlimFL: Federated Learning with Superposition Coding over Slimmable
  Neural Networks
  Federated learning (FL) is a key enabler for efficient communication and
computing leveraging devices' distributed computing capabilities. However,
applying FL in practice is challenging due to the local devices' heterogeneous
energy, wireless channel conditions, and non-independently and identically
distributed (non-IID) data distributions. To cope with these issues, this paper
proposes a novel learning framework by integrating FL and width-adjustable
slimmable neural networks (SNN). Integrating FL with SNNs is challenging due to
time-varing channel conditions and data distributions. In addition, existing
multi-width SNN training algorithms are sensitive to the data distributions
across devices, which makes SNN ill-suited for FL. Motivated by this, we
propose a communication and energy-efficient SNN-based FL (named SlimFL) that
jointly utilizes superposition coding (SC) for global model aggregation and
superposition training (ST) for updating local models. By applying SC, SlimFL
exchanges the superposition of multiple width configurations decoded as many
times as possible for a given communication throughput. Leveraging ST, SlimFL
aligns the forward propagation of different width configurations while avoiding
inter-width interference during backpropagation. We formally prove the
convergence of SlimFL. The result reveals that SlimFL is not only
communication-efficient but also deals with the non-IID data distributions and
poor channel conditions, which is also corroborated by data-intensive
simulations.
