Federated Few-Shot Learning with Adversarial Learning
  We are interested in developing a unified machine learning model over many
mobile devices for practical learning tasks, where each device only has very
few training data. This is a commonly encountered situation in mobile computing
scenarios, where data is scarce and distributed while the tasks are distinct.
In this paper, we propose a federated few-shot learning (FedFSL) framework to
learn a few-shot classification model that can classify unseen data classes
with only a few labeled samples. With the federated learning strategy, FedFSL
can utilize many data sources while keeping data privacy and communication
efficiency. There are two technical challenges: 1) directly using the existing
federated learning approach may lead to misaligned decision boundaries produced
by client models, and 2) constraining the decision boundaries to be similar
over clients would overfit to training tasks but not adapt well to unseen
tasks. To address these issues, we propose to regularize local updates by
minimizing the divergence of client models. We also formulate the training in
an adversarial fashion and optimize the client models to produce a
discriminative feature space that can better represent unseen data samples. We
demonstrate the intuitions and conduct experiments to show our approaches
outperform baselines by more than 10% in learning vision tasks and 5% in
language tasks.
