HeteroFL: Computation and Communication Efficient Federated Learning for
  Heterogeneous Clients
  Federated Learning (FL) is a method of training machine learning models on
private data distributed over a large number of possibly heterogeneous clients
such as mobile phones and IoT devices. In this work, we propose a new federated
learning framework named HeteroFL to address heterogeneous clients equipped
with very different computation and communication capabilities. Our solution
can enable the training of heterogeneous local models with varying computation
complexities and still produce a single global inference model. For the first
time, our method challenges the underlying assumption of existing work that
local models have to share the same architecture as the global model. We
demonstrate several strategies to enhance FL training and conduct extensive
empirical evaluations, including five computation complexity levels of three
model architecture on three datasets. We show that adaptively distributing
subnetworks according to clients' capabilities is both computation and
communication efficient.
