EMPIR: Ensembles of Mixed Precision Deep Networks for Increased
  Robustness against Adversarial Attacks
  Ensuring robustness of Deep Neural Networks (DNNs) is crucial to their
adoption in safety-critical applications such as self-driving cars, drones, and
healthcare. Notably, DNNs are vulnerable to adversarial attacks in which small
input perturbations can produce catastrophic misclassifications. In this work,
we propose EMPIR, ensembles of quantized DNN models with different numerical
precisions, as a new approach to increase robustness against adversarial
attacks. EMPIR is based on the observation that quantized neural networks often
demonstrate much higher robustness to adversarial attacks than full precision
networks, but at the cost of a substantial loss in accuracy on the original
(unperturbed) inputs. EMPIR overcomes this limitation to achieve the 'best of
both worlds', i.e., the higher unperturbed accuracies of the full precision
models combined with the higher robustness of the low precision models, by
composing them in an ensemble. Further, as low precision DNN models have
significantly lower computational and storage requirements than full precision
models, EMPIR models only incur modest compute and memory overheads compared to
a single full-precision model (<25% in our evaluations). We evaluate EMPIR
across a suite of DNNs for 3 different image recognition tasks (MNIST, CIFAR-10
and ImageNet) and under 4 different adversarial attacks. Our results indicate
that EMPIR boosts the average adversarial accuracies by 42.6%, 15.2% and 10.5%
for the DNN models trained on the MNIST, CIFAR-10 and ImageNet datasets
respectively, when compared to single full-precision models, without
sacrificing accuracy on the unperturbed inputs.
