QUOTIENT: Two-Party Secure Neural Network Training and Prediction
  Recently, there has been a wealth of effort devoted to the design of secure
protocols for machine learning tasks. Much of this is aimed at enabling secure
prediction from highly-accurate Deep Neural Networks (DNNs). However, as DNNs
are trained on data, a key question is how such models can be also trained
securely. The few prior works on secure DNN training have focused either on
designing custom protocols for existing training algorithms, or on developing
tailored training algorithms and then applying generic secure protocols. In
this work, we investigate the advantages of designing training algorithms
alongside a novel secure protocol, incorporating optimizations on both fronts.
We present QUOTIENT, a new method for discretized training of DNNs, along with
a customized secure two-party protocol for it. QUOTIENT incorporates key
components of state-of-the-art DNN training such as layer normalization and
adaptive gradient methods, and improves upon the state-of-the-art in DNN
training in two-party computation. Compared to prior work, we obtain an
improvement of 50X in WAN time and 6% in absolute accuracy.
