Secure Federated Submodel Learning
  Federated learning was proposed with an intriguing vision of achieving
collaborative machine learning among numerous clients without uploading their
private data to a cloud server. However, the conventional framework requires
each client to leverage the full model for learning, which can be prohibitively
inefficient for resource-constrained clients and large-scale deep learning
tasks. We thus propose a new framework, called federated submodel learning,
where clients download only the needed parts of the full model, namely
submodels, and then upload the submodel updates. Nevertheless, the "position"
of a client's truly required submodel corresponds to her private data, and its
disclosure to the cloud server during interactions inevitably breaks the tenet
of federated learning. To integrate efficiency and privacy, we have designed a
secure federated submodel learning scheme coupled with a private set union
protocol as a cornerstone. Our secure scheme features the properties of
randomized response, secure aggregation, and Bloom filter, and endows each
client with a customized plausible deniability, in terms of local differential
privacy, against the position of her desired submodel, thus protecting her
private data. We further instantiated our scheme with the e-commerce
recommendation scenario in Alibaba, implemented a prototype system, and
extensively evaluated its performance over 30-day Taobao user data. The
analysis and evaluation results demonstrate the feasibility and scalability of
our scheme from model accuracy and convergency, practical communication,
computation, and storage overheads, as well as manifest its remarkable
advantages over the conventional federated learning framework.
