Over-the-Air Federated Multi-Task Learning via Model Sparsification and Turbo Compressed Sensing
To achieve communication-efficient federated multitask learning (FMTL), we
propose an over-the-air FMTL (OAFMTL) framework, where multiple learning tasks
deployed on edge devices share a non-orthogonal fading channel under the
coordination of an edge server (ES). In OA-FMTL, the local updates of edge
devices are sparsified, compressed, and then sent over the uplink channel in a
superimposed fashion. The ES employs over-the-air computation in the presence
of intertask interference. More specifically, the model aggregations of all the
tasks are reconstructed from the channel observations concurrently, based on a
modified version of the turbo compressed sensing (Turbo-CS) algorithm (named as
M-Turbo-CS). We analyze the performance of the proposed OA-FMTL framework
together with the M-Turbo-CS algorithm. Furthermore, based on the analysis, we
formulate a communication-learning optimization problem to improve the system
performance by adjusting the power allocation among the tasks at the edge
devices. Numerical simulations show that our proposed OAFMTL effectively
suppresses the inter-task interference, and achieves a learning performance
comparable to its counterpart with orthogonal multi-task transmission. It is
also shown that the proposed inter-task power allocation optimization algorithm
substantially reduces the overall communication overhead by appropriately
adjusting the power allocation among the tasks.