PrivLogit: Efficient Privacy-preserving Logistic Regression by Tailoring
  Numerical Optimizers
  Safeguarding privacy in machine learning is highly desirable, especially in
collaborative studies across many organizations. Privacy-preserving distributed
machine learning (based on cryptography) is popular to solve the problem.
However, existing cryptographic protocols still incur excess computational
overhead. Here, we make a novel observation that this is partially due to naive
adoption of mainstream numerical optimization (e.g., Newton method) and failing
to tailor for secure computing. This work presents a contrasting perspective:
customizing numerical optimization specifically for secure settings. We propose
a seemingly less-favorable optimization method that can in fact significantly
accelerate privacy-preserving logistic regression. Leveraging this new method,
we propose two new secure protocols for conducting logistic regression in a
privacy-preserving and distributed manner. Extensive theoretical and empirical
evaluations prove the competitive performance of our two secure proposals while
without compromising accuracy or privacy: with speedup up to 2.3x and 8.1x,
respectively, over state-of-the-art; and even faster as data scales up. Such
drastic speedup is on top of and in addition to performance improvements from
existing (and future) state-of-the-art cryptography. Our work provides a new
way towards efficient and practical privacy-preserving logistic regression for
large-scale studies which are common for modern science.
