Adaptive Subgradient Methods for Online AUC Maximization
  Learning for maximizing AUC performance is an important research problem in
Machine Learning and Artificial Intelligence. Unlike traditional batch learning
methods for maximizing AUC which often suffer from poor scalability, recent
years have witnessed some emerging studies that attempt to maximize AUC by
single-pass online learning approaches. Despite their encouraging results
reported, the existing online AUC maximization algorithms often adopt simple
online gradient descent approaches that fail to exploit the geometrical
knowledge of the data observed during the online learning process, and thus
could suffer from relatively larger regret. To address the above limitation, in
this work, we explore a novel algorithm of Adaptive Online AUC Maximization
(AdaOAM) which employs an adaptive gradient method that exploits the knowledge
of historical gradients to perform more informative online learning. The new
adaptive updating strategy of the AdaOAM is less sensitive to the parameter
settings and maintains the same time complexity as previous non-adaptive
counterparts. Additionally, we extend the algorithm to handle high-dimensional
sparse data (SAdaOAM) and address sparsity in the solution by performing lazy
gradient updating. We analyze the theoretical bounds and evaluate their
empirical performance on various types of data sets. The encouraging empirical
results obtained clearly highlighted the effectiveness and efficiency of the
proposed algorithms.
