Calibrated Surrogate Losses for Adversarially Robust Classification
  Adversarially robust classification seeks a classifier that is insensitive to
adversarial perturbations of test patterns. This problem is often formulated
via a minimax objective, where the target loss is the worst-case value of the
0-1 loss subject to a bound on the size of perturbation. Recent work has
proposed convex surrogates for the adversarial 0-1 loss, in an effort to make
optimization more tractable. A primary question is that of consistency, that
is, whether minimization of the surrogate risk implies minimization of the
adversarial 0-1 risk. In this work, we analyze this question through the lens
of calibration, which is a pointwise notion of consistency. We show that no
convex surrogate loss is calibrated with respect to the adversarial 0-1 loss
when restricted to the class of linear models. We further introduce a class of
nonconvex losses and offer necessary and sufficient conditions for losses in
this class to be calibrated. We also show that if the underlying distribution
satisfies Massart's noise condition, convex losses can also be calibrated in
the adversarial setting.
