Self-supervised Cross-silo Federated Neural Architecture Search
  Federated Learning (FL) provides both model performance and data privacy for
machine learning tasks where samples or features are distributed among
different parties. In the training process of FL, no party has a global view of
data distributions or model architectures of other parties. Thus the
manually-designed architectures may not be optimal. In the past, Neural
Architecture Search (NAS) has been applied to FL to address this critical
issue. However, existing Federated NAS approaches require prohibitive
communication and computation effort, as well as the availability of
high-quality labels. In this work, we present Self-supervised Vertical
Federated Neural Architecture Search (SS-VFNAS) for automating FL where
participants hold feature-partitioned data, a common cross-silo scenario called
Vertical Federated Learning (VFL). In the proposed framework, each party first
conducts NAS using self-supervised approach to find a local optimal
architecture with its own data. Then, parties collaboratively improve the local
optimal architecture in a VFL framework with supervision. We demonstrate
experimentally that our approach has superior performance, communication
efficiency and privacy compared to Federated NAS and is capable of generating
high-performance and highly-transferable heterogeneous architectures even with
insufficient overlapping samples, providing automation for those parties
without deep learning expertise.
