Adversarial AutoAugment
  Data augmentation (DA) has been widely utilized to improve generalization in
training deep neural networks. Recently, human-designed data augmentation has
been gradually replaced by automatically learned augmentation policy. Through
finding the best policy in well-designed search space of data augmentation,
AutoAugment can significantly improve validation accuracy on image
classification tasks. However, this approach is not computationally practical
for large-scale problems. In this paper, we develop an adversarial method to
arrive at a computationally-affordable solution called Adversarial AutoAugment,
which can simultaneously optimize target related object and augmentation policy
search loss. The augmentation policy network attempts to increase the training
loss of a target network through generating adversarial augmentation policies,
while the target network can learn more robust features from harder examples to
improve the generalization. In contrast to prior work, we reuse the computation
in target network training for policy evaluation, and dispense with the
retraining of the target network. Compared to AutoAugment, this leads to about
12x reduction in computing cost and 11x shortening in time overhead on
ImageNet. We show experimental results of our approach on CIFAR-10/CIFAR-100,
ImageNet, and demonstrate significant performance improvements over
state-of-the-art. On CIFAR-10, we achieve a top-1 test error of 1.36%, which is
the currently best performing single model. On ImageNet, we achieve a leading
performance of top-1 accuracy 79.40% on ResNet-50 and 80.00% on ResNet-50-D
without extra data.
