Informative Dropout for Robust Representation Learning: A Shape-bias
  Perspective
  Convolutional Neural Networks (CNNs) are known to rely more on local texture
rather than global shape when making decisions. Recent work also indicates a
close relationship between CNN's texture-bias and its robustness against
distribution shift, adversarial perturbation, random corruption, etc. In this
work, we attempt at improving various kinds of robustness universally by
alleviating CNN's texture bias. With inspiration from the human visual system,
we propose a light-weight model-agnostic method, namely Informative Dropout
(InfoDrop), to improve interpretability and reduce texture bias. Specifically,
we discriminate texture from shape based on local self-information in an image,
and adopt a Dropout-like algorithm to decorrelate the model output from the
local texture. Through extensive experiments, we observe enhanced robustness
under various scenarios (domain generalization, few-shot classification, image
corruption, and adversarial perturbation). To the best of our knowledge, this
work is one of the earliest attempts to improve different kinds of robustness
in a unified model, shedding new light on the relationship between shape-bias
and robustness, also on new approaches to trustworthy machine learning
algorithms. Code is available at https://github.com/bfshi/InfoDrop.
