Multi-Task Federated Learning for Personalised Deep Neural Networks in
  Edge Computing
  Federated Learning (FL) is an emerging approach for collaboratively training
Deep Neural Networks (DNNs) on mobile devices, without private user data
leaving the devices. Previous works have shown that non-Independent and
Identically Distributed (non-IID) user data harms the convergence speed of the
FL algorithms. Furthermore, most existing work on FL measures global-model
accuracy, but in many cases, such as user content-recommendation, improving
individual User model Accuracy (UA) is the real objective. To address these
issues, we propose a Multi-Task FL (MTFL) algorithm that introduces
non-federated Batch-Normalization (BN) layers into the federated DNN. MTFL
benefits UA and convergence speed by allowing users to train models
personalised to their own data. MTFL is compatible with popular iterative FL
optimisation algorithms such as Federated Averaging (FedAvg), and we show
empirically that a distributed form of Adam optimisation (FedAvg-Adam) benefits
convergence speed even further when used as the optimisation strategy within
MTFL. Experiments using MNIST and CIFAR10 demonstrate that MTFL is able to
significantly reduce the number of rounds required to reach a target UA, by up
to $5\times$ when using existing FL optimisation strategies, and with a further
$3\times$ improvement when using FedAvg-Adam. We compare MTFL to competing
personalised FL algorithms, showing that it is able to achieve the best UA for
MNIST and CIFAR10 in all considered scenarios. Finally, we evaluate MTFL with
FedAvg-Adam on an edge-computing testbed, showing that its convergence and UA
benefits outweigh its overhead.
