Measuring Adversarial Robustness using a Voronoi-Epsilon Adversary
  Previous studies on robustness have argued that there is a tradeoff between
accuracy and adversarial accuracy. The tradeoff can be inevitable even when we
neglect generalization. We argue that the tradeoff is inherent to the commonly
used definition of adversarial accuracy, which uses an adversary that can
construct adversarial points constrained by $\epsilon$-balls around data
points. As $\epsilon$ gets large, the adversary may use real data points from
other classes as adversarial examples. We propose a Voronoi-epsilon adversary
which is constrained both by Voronoi cells and by $\epsilon$-balls. This
adversary balances between two notions of perturbation. As a result,
adversarial accuracy based on this adversary avoids a tradeoff between accuracy
and adversarial accuracy on training data even when $\epsilon$ is large.
Finally, we show that a nearest neighbor classifier is the maximally robust
classifier against the proposed adversary on the training data.
