Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness
  Mode connectivity provides novel geometric insights on analyzing loss
landscapes and enables building high-accuracy pathways between well-trained
neural networks. In this work, we propose to employ mode connectivity in loss
landscapes to study the adversarial robustness of deep neural networks, and
provide novel methods for improving this robustness. Our experiments cover
various types of adversarial attacks applied to different network architectures
and datasets. When network models are tampered with backdoor or error-injection
attacks, our results demonstrate that the path connection learned using limited
amount of bonafide data can effectively mitigate adversarial effects while
maintaining the original accuracy on clean data. Therefore, mode connectivity
provides users with the power to repair backdoored or error-injected models. We
also use mode connectivity to investigate the loss landscapes of regular and
robust models against evasion attacks. Experiments show that there exists a
barrier in adversarial robustness loss on the path connecting regular and
adversarially-trained models. A high correlation is observed between the
adversarial robustness loss and the largest eigenvalue of the input Hessian
matrix, for which theoretical justifications are provided. Our results suggest
that mode connectivity offers a holistic tool and practical means for
evaluating and improving adversarial robustness.
