Adversarially Trained Models with Test-Time Covariate Shift Adaptation
  We empirically demonstrate that test-time adaptive batch normalization, which
re-estimates the batch-normalization statistics during inference, can provide
$\ell_2$-certification as well as improve the commonly occurring corruption
robustness of adversarially trained models while maintaining their
state-of-the-art empirical robustness against adversarial attacks. Furthermore,
we obtain similar $\ell_2$-certification as the current state-of-the-art
certification models for CIFAR-10 by learning our adversarially trained model
using larger $\ell_2$-bounded adversaries. Therefore our work is a step towards
bridging the gap between the state-of-the-art certification and empirical
robustness. Our results also indicate that improving the empirical adversarial
robustness may be sufficient as we achieve certification and corruption
robustness as a by-product using test-time adaptive batch normalization.
