Midpoint Regularization: from High Uncertainty Training to Conservative
  Classification
  Label Smoothing (LS) improves model generalization through penalizing models
from generating overconfident output distributions. For each training sample
the LS strategy smooths the one-hot encoded training signal by distributing its
distribution mass over the non-ground truth classes. We extend this technique
by considering example pairs, coined PLS. PLS first creates midpoint samples by
averaging random sample pairs and then learns a smoothing distribution during
training for each of these midpoint samples, resulting in midpoints with high
uncertainty labels for training. We empirically show that PLS significantly
outperforms LS, achieving up to 30% of relative classification error reduction.
We also visualize that PLS produces very low winning softmax scores for both in
and out of distribution samples.
