Improving Adversarial Robustness for Free with Snapshot Ensemble
  Adversarial training, as one of the few certified defenses against
adversarial attacks, can be quite complicated and time-consuming, while the
results might not be robust enough. To address the issue of lack of robustness,
ensemble methods were proposed, aiming to get the final output by weighting the
selected results from repeatedly trained processes. It is proved to be very
useful in achieving robust and accurate results, but the computational and
memory costs are even higher. Snapshot ensemble, a new ensemble method that
combines several local minima in a single training process to make the final
prediction, was proposed recently, which reduces the time spent on training
multiple networks and the memory to store the results. Based on the snapshot
ensemble, we present a new method that is easier to implement: unlike original
snapshot ensemble that seeks for local minima, our snapshot ensemble focuses on
the last few iterations of a training and stores the sets of parameters from
them. Our algorithm is much simpler but the results are no less accurate than
the original ones: based on different hyperparameters and datasets, our
snapshot ensemble has shown a 5% to 30% increase in accuracy when compared to
the traditional adversarial training.
