SPEED: Secure, PrivatE, and Efficient Deep learning
  We introduce a deep learning framework able to deal with strong privacy
constraints. Based on collaborative learning, differential privacy and
homomorphic encryption, the proposed approach advances state-of-the-art of
private deep learning against a wider range of threats, in particular the
honest-but-curious server assumption. We address threats from both the
aggregation server, the global model and potentially colluding data holders.
Building upon distributed differential privacy and a homomorphic argmax
operator, our method is specifically designed to maintain low communication
loads and efficiency. The proposed method is supported by carefully crafted
theoretical results. We provide differential privacy guarantees from the point
of view of any entity having access to the final model, including colluding
data holders, as a function of the ratio of data holders who kept their noise
secret. This makes our method practical to real-life scenarios where data
holders do not trust any third party to process their datasets nor the other
data holders. Crucially the computational burden of the approach is maintained
reasonable, and, to the best of our knowledge, our framework is the first one
to be efficient enough to investigate deep learning applications while
addressing such a large scope of threats. To assess the practical usability of
our framework, experiments have been carried out on image datasets in a
classification context. We present numerical results that show that the
learning procedure is both accurate and private.
