SPIDER: Searching Personalized Neural Architecture for Federated
  Learning
  Federated learning (FL) is an efficient learning framework that assists
distributed machine learning when data cannot be shared with a centralized
server due to privacy and regulatory restrictions. Recent advancements in FL
use predefined architecture-based learning for all the clients. However, given
that clients' data are invisible to the server and data distributions are
non-identical across clients, a predefined architecture discovered in a
centralized setting may not be an optimal solution for all the clients in FL.
Motivated by this challenge, in this work, we introduce SPIDER, an algorithmic
framework that aims to Search Personalized neural architecture for federated
learning. SPIDER is designed based on two unique features: (1) alternately
optimizing one architecture-homogeneous global model (Supernet) in a generic FL
manner and one architecture-heterogeneous local model that is connected to the
global model by weight sharing-based regularization (2) achieving
architecture-heterogeneous local model by a novel neural architecture search
(NAS) method that can select optimal subnet progressively using operation-level
perturbation on the accuracy value as the criterion. Experimental results
demonstrate that SPIDER outperforms other state-of-the-art personalization
methods, and the searched personalized architectures are more inference
efficient.
