Quadruply Stochastic Gradient Method for Large Scale Nonlinear
  Semi-Supervised Ordinal Regression AUC Optimization
  Semi-supervised ordinal regression (S$^2$OR) problems are ubiquitous in
real-world applications, where only a few ordered instances are labeled and
massive instances remain unlabeled. Recent researches have shown that directly
optimizing concordance index or AUC can impose a better ranking on the data
than optimizing the traditional error rate in ordinal regression (OR) problems.
In this paper, we propose an unbiased objective function for S$^2$OR AUC
optimization based on ordinal binary decomposition approach. Besides, to handle
the large-scale kernelized learning problems, we propose a scalable algorithm
called QS$^3$ORAO using the doubly stochastic gradients (DSG) framework for
functional optimization. Theoretically, we prove that our method can converge
to the optimal solution at the rate of $O(1/t)$, where $t$ is the number of
iterations for stochastic data sampling. Extensive experimental results on
various benchmark and real-world datasets also demonstrate that our method is
efficient and effective while retaining similar generalization performance.
