Privacy-Preserving Logistic Regression Training with a Faster Gradient
  Variant
  Logistic regression training on an encrypted dataset has been an attractive
idea to security concerns for years. In this paper, we propose a faster
gradient variant called Quadratic Gradient for logistic regression and
implement it via a special homomorphic encryption scheme. The core of this
gradient variant can be seen as an extension of the simplified fixed Hessian
from Newton's method, which extracts information from the Hessian matrix into
the naive gradient, and thus can be used to enhance Nesterov's accelerated
gradient (NAG), Adagrad, etc. We evaluate various gradient $ascent$ methods
with this gradient variant on the gene dataset provided by the 2017 iDASH
competition and the image dataset from the MNIST database. Experimental results
show that the enhanced methods converge faster and sometimes even to a better
convergence result. We also implement the gradient variant in full batch NAG
and mini-batch NAG for training a logistic regression model on a large dataset
in the encrypted domain. Equipped with this gradient variant, full batch NAG
and mini-batch NAG are both faster than the original ones.
