On Mixup Regularization
  Mixup is a data augmentation technique that creates new examples as convex
combinationsof training points and labels. This simple technique has
empirically shown to improvethe accuracy of many state-of-the-art models in
different settings and applications, butthe reasons behind this empirical
success remain poorly understood. In this paper wetake a substantial step in
explaining the theoretical foundations of Mixup, by clarifyingits
regularization effects. We show that Mixup can be interpreted as standard
empiricalrisk minimization estimator subject to a combination of data
transformation and randomperturbation of the transformed data. We gain two core
insights from this new interpretation.First, the data transformation suggests
that, at test time, a model trained with Mixup shouldalso be applied to
transformed data, a one-line change in code that we show empirically toimprove
both accuracy and calibration of the prediction. Second, we show how the
randomperturbation of the new interpretation of Mixup induces multiple known
regularizationschemes, including label smoothing and reduction of the Lipschitz
constant of the estimator.These schemes interact synergistically with each
other, resulting in a self calibrated andeffective regularization effect that
prevents overfitting and overconfident predictions. Wecorroborate our
theoretical analysis with experiments that support our conclusions.
