Optimizing Two-way Partial AUC with an End-to-end Framework
The Area Under the ROC Curve (AUC) is a crucial metric for machine learning,
which evaluates the average performance over all possible True Positive Rates
(TPRs) and False Positive Rates (FPRs). Based on the knowledge that a skillful
classifier should simultaneously embrace a high TPR and a low FPR, we turn to
study a more general variant called Two-way Partial AUC (TPAUC), where only the
region with $\mathsf{TPR} \ge \alpha, \mathsf{FPR} \le \beta$ is included in
the area. Moreover, recent work shows that the TPAUC is essentially
inconsistent with the existing Partial AUC metrics where only the FPR range is
restricted, opening a new problem to seek solutions to leverage high TPAUC.
Motivated by this, we present the first trial in this paper to optimize this
new metric. The critical challenge along this course lies in the difficulty of
performing gradient-based optimization with end-to-end stochastic training,
even with a proper choice of surrogate loss. To address this issue, we propose
a generic framework to construct surrogate optimization problems, which
supports efficient end-to-end training with deep learning. Moreover, our
theoretical analyses show that: 1) the objective function of the surrogate
problems will achieve an upper bound of the original problem under mild
conditions, and 2) optimizing the surrogate problems leads to good
generalization performance in terms of TPAUC with a high probability. Finally,
empirical studies over several benchmark datasets speak to the efficacy of our
framework.