Adversarial Training Makes Weight Loss Landscape Sharper in Logistic
  Regression
  Adversarial training is actively studied for learning robust models against
adversarial examples. A recent study finds that adversarially trained models
degenerate generalization performance on adversarial examples when their weight
loss landscape, which is loss changes with respect to weights, is sharp.
Unfortunately, it has been experimentally shown that adversarial training
sharpens the weight loss landscape, but this phenomenon has not been
theoretically clarified. Therefore, we theoretically analyze this phenomenon in
this paper. As a first step, this paper proves that adversarial training with
the L2 norm constraints sharpens the weight loss landscape in the linear
logistic regression model. Our analysis reveals that the sharpness of the
weight loss landscape is caused by the noise aligned in the direction of
increasing the loss, which is used in adversarial training. We theoretically
and experimentally confirm that the weight loss landscape becomes sharper as
the magnitude of the noise of adversarial training increases in the linear
logistic regression model. Moreover, we experimentally confirm the same
phenomena in ResNet18 with softmax as a more general case.
