Evaluating the Robustness of Geometry-Aware Instance-Reweighted
  Adversarial Training
  In this technical report, we evaluate the adversarial robustness of a very
recent method called "Geometry-aware Instance-reweighted Adversarial
Training"[7]. GAIRAT reports state-of-the-art results on defenses to
adversarial attacks on the CIFAR-10 dataset. In fact, we find that a network
trained with this method, while showing an improvement over regular adversarial
training (AT), is biasing the model towards certain samples by re-scaling the
loss. Indeed, this leads the model to be susceptible to attacks that scale the
logits. The original model shows an accuracy of 59% under AutoAttack - when
trained with additional data with pseudo-labels. We provide an analysis that
shows the opposite. In particular, we craft a PGD attack multiplying the logits
by a positive scalar that decreases the GAIRAT accuracy from from 55% to 44%,
when trained solely on CIFAR-10. In this report, we rigorously evaluate the
model and provide insights into the reasons behind the vulnerability of GAIRAT
to this adversarial attack. The code to reproduce our evaluation is made
available at https://github.com/giuxhub/GAIRAT-LSA
