StackMix: A complementary Mix algorithm
  Techniques combining multiple images as input/output have proven to be
effective data augmentations for training convolutional neural networks. In
this paper, we present StackMix: Each input is presented as a concatenation of
two images, and the label is the mean of the two one-hot labels. On its own,
StackMix rivals other widely used methods in the "Mix" line of work. More
importantly, unlike previous work, significant gains across a variety of
benchmarks are achieved by combining StackMix with existing Mix augmentation,
effectively mixing more than two images. E.g., by combining StackMix with
CutMix, test error in the supervised setting is improved across a variety of
settings over CutMix, including 0.8\% on ImageNet, 3\% on Tiny ImageNet, 2\% on
CIFAR-100, 0.5\% on CIFAR-10, and 1.5\% on STL-10. Similar results are achieved
with Mixup.We further show that gains hold for robustness to common input
corruptions and perturbations at varying severities with a 0.7\% improvement on
CIFAR-100-C, by combining StackMix with AugMix over AugMix. On its own,
improvements with StackMix hold across different number of labeled samples on
CIFAR-100, maintaining approximately a 2\% gap in test accuracy -- down to
using only 5\% of the whole dataset -- and is effective in the semi-supervised
setting with a 2\% improvement with the standard benchmark $\Pi$-model.
Finally, we perform an extensive ablation study to better understand the
proposed algorithm.
