An SDE Framework for Adversarial Training, with Convergence and
  Robustness Analysis
  Adversarial training has gained great popularity as one of the most effective
defenses for deep neural networks against adversarial perturbations on data
points. Consequently, research interests have grown in understanding the
convergence and robustness of adversarial training. This paper considers the
min-max game of adversarial training by alternating stochastic gradient
descent. It approximates the training process with a continuous-time
stochastic-differential-equation (SDE). In particular, the error bound and
convergence analysis is established.
  This SDE framework allows direct comparison between adversarial training and
stochastic gradient descent; and confirms analytically the robustness of
adversarial training from a (new) gradient-flow viewpoint. This analysis is
then corroborated via numerical studies.
  To demonstrate the versatility of this SDE framework for algorithm design and
parameter tuning, a stochastic control problem is formulated for learning rate
adjustment, where the advantage of adaptive learning rate over fixed learning
rate in terms of training loss is demonstrated through numerical experiments.
