Privacy-Preserving Asynchronous Federated Learning Algorithms for
  Multi-Party Vertically Collaborative Learning
  The privacy-preserving federated learning for vertically partitioned data has
shown promising results as the solution of the emerging multi-party joint
modeling application, in which the data holders (such as government branches,
private finance and e-business companies) collaborate throughout the learning
process rather than relying on a trusted third party to hold data. However,
existing federated learning algorithms for vertically partitioned data are
limited to synchronous computation. To improve the efficiency when the
unbalanced computation/communication resources are common among the parties in
the federated learning system, it is essential to develop asynchronous training
algorithms for vertically partitioned data while keeping the data privacy. In
this paper, we propose an asynchronous federated SGD (AFSGD-VP) algorithm and
its SVRG and SAGA variants on the vertically partitioned data. Moreover, we
provide the convergence analyses of AFSGD-VP and its SVRG and SAGA variants
under the condition of strong convexity. We also discuss their model privacy,
data privacy, computational complexities and communication costs. To the best
of our knowledge, AFSGD-VP and its SVRG and SAGA variants are the first
asynchronous federated learning algorithms for vertically partitioned data.
Extensive experimental results on a variety of vertically partitioned datasets
not only verify the theoretical results of AFSGD-VP and its SVRG and SAGA
variants, but also show that our algorithms have much higher efficiency than
the corresponding synchronous algorithms.
