Enhancing Intrinsic Adversarial Robustness via Feature Pyramid Decoder
  Whereas adversarial training is employed as the main defence strategy against
specific adversarial samples, it has limited generalization capability and
incurs excessive time complexity. In this paper, we propose an attack-agnostic
defence framework to enhance the intrinsic robustness of neural networks,
without jeopardizing the ability of generalizing clean samples. Our Feature
Pyramid Decoder (FPD) framework applies to all block-based convolutional neural
networks (CNNs). It implants denoising and image restoration modules into a
targeted CNN, and it also constraints the Lipschitz constant of the
classification layer. Moreover, we propose a two-phase strategy to train the
FPD-enhanced CNN, utilizing $\epsilon$-neighbourhood noisy images with
multi-task and self-supervised learning. Evaluated against a variety of
white-box and black-box attacks, we demonstrate that FPD-enhanced CNNs gain
sufficient robustness against general adversarial samples on MNIST, SVHN and
CALTECH. In addition, if we further conduct adversarial training, the
FPD-enhanced CNNs perform better than their non-enhanced versions.
