When Should You Defend Your Classifier -- A Game-theoretical Analysis of
  Countermeasures against Adversarial Examples
  Adversarial machine learning, i.e., increasing the robustness of machine
learning algorithms against so-called adversarial examples, is now an
established field. Yet, newly proposed methods are evaluated and compared under
unrealistic scenarios where costs for adversary and defender are not considered
and either all samples or no samples are adversarially perturbed. We scrutinize
these assumptions and propose the advanced adversarial classification game,
which incorporates all relevant parameters of an adversary and a defender.
Especially, we take into account economic factors on both sides and the fact
that all so far proposed countermeasures against adversarial examples reduce
accuracy on benign samples. Analyzing the scenario in detail, where both
players have two pure strategies, we identify all best responses and conclude
that in practical settings, the most influential factor might be the maximum
amount of adversarial examples.
