Robustness May Be at Odds with Fairness: An Empirical Study on
  Class-wise Accuracy
  Convolutional neural networks (CNNs) have made significant advancement,
however, they are widely known to be vulnerable to adversarial attacks.
Adversarial training is the most widely used technique for improving
adversarial robustness to strong white-box attacks. Prior works have been
evaluating and improving the model average robustness without class-wise
evaluation. The average evaluation alone might provide a false sense of
robustness. For example, the attacker can focus on attacking the vulnerable
class, which can be dangerous, especially, when the vulnerable class is a
critical one, such as "human" in autonomous driving. We propose an empirical
study on the class-wise accuracy and robustness of adversarially trained
models. We find that there exists inter-class discrepancy for accuracy and
robustness even when the training dataset has an equal number of samples for
each class. For example, in CIFAR10, "cat" is much more vulnerable than other
classes. Moreover, this inter-class discrepancy also exists for normally
trained models, while adversarial training tends to further increase the
discrepancy. Our work aims to investigate the following questions: (a) is the
phenomenon of inter-class discrepancy universal regardless of datasets, model
architectures and optimization hyper-parameters? (b) If so, what can be
possible explanations for the inter-class discrepancy? (c) Can the techniques
proposed in the long tail classification be readily extended to adversarial
training for addressing the inter-class discrepancy?
