Implicit Rate-Constrained Optimization of Non-decomposable Objectives
  We consider a popular family of constrained optimization problems arising in
machine learning that involve optimizing a non-decomposable evaluation metric
with a certain thresholded form, while constraining another metric of interest.
Examples of such problems include optimizing the false negative rate at a fixed
false positive rate, optimizing precision at a fixed recall, optimizing the
area under the precision-recall or ROC curves, etc. Our key idea is to
formulate a rate-constrained optimization that expresses the threshold
parameter as a function of the model parameters via the Implicit Function
theorem. We show how the resulting optimization problem can be solved using
standard gradient based methods. Experiments on benchmark datasets demonstrate
the effectiveness of our proposed method over existing state-of-the art
approaches for these problems. The code for the proposed method is available at
https://github.com/google-research/google-research/tree/master/implicit_constrained_optimization .
