Privacy-Preserving Distributed Learning with Secret Gradient Descent
  In many important application domains of machine learning, data is a
privacy-sensitive resource. In addition, due to the growing complexity of the
models, single actors typically do not have sufficient data to train a model on
their own. Motivated by these challenges, we propose Secret Gradient Descent
(SecGD), a method for training machine learning models on data that is spread
over different clients while preserving the privacy of the training data. We
achieve this by letting each client add temporary noise to the information they
send to the server during the training process. They also share this noise in
separate messages with the server, which can then subtract it from the
previously received values. By routing all data through an anonymization
network such as Tor, we prevent the server from knowing which messages
originate from the same client, which in turn allows us to show that breaking a
client's privacy is computationally intractable as it would require solving a
hard instance of the subset sum problem. This setup allows SecGD to work in the
presence of only two honest clients and a malicious server, and without the
need for peer-to-peer connections.
