SLOE: A Faster Method for Statistical Inference in High-Dimensional
  Logistic Regression
  Logistic regression remains one of the most widely used tools in applied
statistics, machine learning and data science. However, in moderately
high-dimensional problems, where the number of features $d$ is a non-negligible
fraction of the sample size $n$, the logistic regression maximum likelihood
estimator (MLE), and statistical procedures based the large-sample
approximation of its distribution, behave poorly. Recently, Sur and Cand\`es
(2019) showed that these issues can be corrected by applying a new
approximation of the MLE's sampling distribution in this high-dimensional
regime. Unfortunately, these corrections are difficult to implement in
practice, because they require an estimate of the \emph{signal strength}, which
is a function of the underlying parameters $\beta$ of the logistic regression.
To address this issue, we propose SLOE, a fast and straightforward approach to
estimate the signal strength in logistic regression. The key insight of SLOE is
that the Sur and Cand\`es (2019) correction can be reparameterized in terms of
the \emph{corrupted signal strength}, which is only a function of the estimated
parameters $\widehat \beta$. We propose an estimator for this quantity, prove
that it is consistent in the relevant high-dimensional regime, and show that
dimensionality correction using SLOE is accurate in finite samples. Compared to
the existing ProbeFrontier heuristic, SLOE is conceptually simpler and orders
of magnitude faster, making it suitable for routine use. We demonstrate the
importance of routine dimensionality correction in the Heart Disease dataset
from the UCI repository, and a genomics application using data from the UK
Biobank. We provide an open source package for this method, available at
\url{https://github.com/google-research/sloe-logistic}.
