Using Multilevel Circulant Matrix Approximation to Speed Up Kernel
  Logistic Regression
  Kernel logistic regression (KLR) is a conventional nonlinear classifier in
machine learning. With the explosive growth of data size, the storage and
computation of large dense kernel matrices is a major challenge in scaling KLR
to massive datasets. This paper proposes an approximate Newton method for
efficiently solving large-scale KLR problems by exploiting the storage and
computing advantages of multilevel circulant matrix(MCM). We reduce the storage
space of the Newton method to $O(n)$ by approximating the kernel matrix
$\boldsymbol K$ with the MCM $\boldsymbol K_{\boldsymbol{q}}$, where $n$ is the
number of training instances and $\boldsymbol{q}$ is the order of the MCM
$\boldsymbol K_{\boldsymbol{q}}$. And by approximating the coefficient matrix
of the Newton equation as an MCM, we can reduce the computational complexity in
the Newton direction to $O(n \log n)$. Our method can run in linearithmic time
at each iteration by using the multidimensional fast Fourier transform (mFFT).
Experimental results on some large-scale binary and multi-classification
problems show that our method enables KLR to scale to large scale problems with
less memory consumption and less time without sacrificing test accuracy.
