Data Selection for Efficient Model Update in Federated Learning
  The Federated Learning (FL) workflow of training a centralized model with
distributed data is growing in popularity. However, until recently, this was
the realm of contributing clients with similar computing capability. The fast
expanding IoT space and data being generated and processed at the edge are
encouraging more effort into expanding federated learning to include
heterogeneous systems. Previous approaches distribute light-weight models to
clients are rely on knowledge transfer to distil the characteristic of local
data in partitioned updates. However, their additional knowledge exchange
transmitted through the network degrades the communication efficiency of FL. We
propose to reduce the size of knowledge exchanged in these FL setups by
clustering and selecting only the most representative bits of information from
the clients. The partitioned global update adopted in our work splits the
global deep neural network into a lower part for generic feature extraction and
an upper part that is more sensitive to this selected client knowledge. Our
experiments show that only 1.6% of the initially exchanged data can effectively
transfer the characteristic of the client data to the global model in our FL
approach, using split networks. These preliminary results evolve our
understanding of federated learning by demonstrating efficient training using
strategically selected training samples.
