On the Inherent Regularization Effects of Noise Injection During
  Training
  Randomly perturbing networks during the training process is a commonly used
approach to improving generalization performance. In this paper, we present a
theoretical study of one particular way of random perturbation, which
corresponds to injecting artificial noise to the training data. We provide a
precise asymptotic characterization of the training and generalization errors
of such randomly perturbed learning problems on a random feature model. Our
analysis shows that Gaussian noise injection in the training process is
equivalent to introducing a weighted ridge regularization, when the number of
noise injections tends to infinity. The explicit form of the regularization is
also given. Numerical results corroborate our asymptotic predictions, showing
that they are accurate even in moderate problem dimensions. Our theoretical
predictions are based on a new correlated Gaussian equivalence conjecture that
generalizes recent results in the study of random feature models.
