Federated Continual Learning with Weighted Inter-client Transfer
  There has been a surge of interest in continual learning and federated
learning, both of which are important in deep neural networks in real-world
scenarios. Yet little research has been done regarding the scenario where each
client learns on a sequence of tasks from a private local data stream. This
problem of federated continual learning poses new challenges to continual
learning, such as utilizing knowledge from other clients, while preventing
interference from irrelevant knowledge. To resolve these issues, we propose a
novel federated continual learning framework, Federated Weighted Inter-client
Transfer (FedWeIT), which decomposes the network weights into global federated
parameters and sparse task-specific parameters, and each client receives
selective knowledge from other clients by taking a weighted combination of
their task-specific parameters. FedWeIT minimizes interference between
incompatible tasks, and also allows positive knowledge transfer across clients
during learning. We validate our FedWeIT against existing federated learning
and continual learning methods under varying degrees of task similarity across
clients, and our model significantly outperforms them with a large reduction in
the communication cost. Code is available at https://github.com/wyjeong/FedWeIT
