Ensemble Defense with Data Diversity: Weak Correlation Implies Strong
  Robustness
  In this paper, we propose a framework of filter-based ensemble of deep
neuralnetworks (DNNs) to defend against adversarial attacks. The framework
builds an ensemble of sub-models -- DNNs with differentiated preprocessing
filters. From the theoretical perspective of DNN robustness, we argue that
under the assumption of high quality of the filters, the weaker the
correlations of the sensitivity of the filters are, the more robust the
ensemble model tends to be, and this is corroborated by the experiments of
transfer-based attacks. Correspondingly, we propose a principle that chooses
the specific filters with smaller Pearson correlation coefficients, which
ensures the diversity of the inputs received by DNNs, as well as the
effectiveness of the entire framework against attacks. Our ensemble models are
more robust than those constructed by previous defense methods like adversarial
training, and even competitive with the classical ensemble of adversarial
trained DNNs under adversarial attacks when the attacking radius is large.
