Using Intuition from Empirical Properties to Simplify Adversarial
  Training Defense
  Due to the surprisingly good representation power of complex distributions,
neural network (NN) classifiers are widely used in many tasks which include
natural language processing, computer vision and cyber security. In recent
works, people noticed the existence of adversarial examples. These adversarial
examples break the NN classifiers' underlying assumption that the environment
is attack free and can easily mislead fully trained NN classifier without
noticeable changes. Among defensive methods, adversarial training is a popular
choice. However, original adversarial training with single-step adversarial
examples (Single-Adv) can not defend against iterative adversarial examples.
Although adversarial training with iterative adversarial examples (Iter-Adv)
can defend against iterative adversarial examples, it consumes too much
computational power and hence is not scalable. In this paper, we analyze
Iter-Adv techniques and identify two of their empirical properties. Based on
these properties, we propose modifications which enhance Single-Adv to perform
competitively as Iter-Adv. Through preliminary evaluation, we show that the
proposed method enhances the test accuracy of state-of-the-art (SOTA)
Single-Adv defensive method against iterative adversarial examples by up to
16.93% while reducing its training cost by 28.75%.
