Robust and Generalizable Visual Representation Learning via Random
  Convolutions
  While successful for various computer vision tasks, deep neural networks have
shown to be vulnerable to texture style shifts and small perturbations to which
humans are robust. In this work, we show that the robustness of neural networks
can be greatly improved through the use of random convolutions as data
augmentation. Random convolutions are approximately shape-preserving and may
distort local textures. Intuitively, randomized convolutions create an infinite
number of new domains with similar global shapes but random local textures.
Therefore, we explore using outputs of multi-scale random convolutions as new
images or mixing them with the original images during training. When applying a
network trained with our approach to unseen domains, our method consistently
improves the performance on domain generalization benchmarks and is scalable to
ImageNet. In particular, in the challenging scenario of generalizing to the
sketch domain in PACS and to ImageNet-Sketch, our method outperforms
state-of-art methods by a large margin. More interestingly, our method can
benefit downstream tasks by providing a more robust pretrained visual
representation.
