Salient ImageNet: How to discover spurious features in Deep Learning?
  Deep neural networks can be unreliable in the real world especially when they
heavily use {\it spurious} features for their predictions. Focusing on image
classifications, we define {\it core features} as the set of visual features
that are always a part of the object definition while {\it spurious features}
are the ones that are likely to {\it co-occur} with the object but not a part
of it (e.g., attribute "fingers" for class "band aid"). Traditional methods for
discovering spurious features either require extensive human annotations (thus,
not scalable), or are useful on specific models. In this work, we introduce a
{\it general} framework to discover a subset of spurious and core visual
features used in inferences of a general model and localize them on a large
number of images with minimal human supervision. Our methodology is based on
this key idea: to identify spurious or core \textit{visual features} used in
model predictions, we identify spurious or core \textit{neural features}
(penultimate layer neurons of a robust model) via limited human supervision
(e.g., using top 5 activating images per feature). We then show that these
neural feature annotations {\it generalize} extremely well to many more images
{\it without} any human supervision. We use the activation maps for these
neural features as the soft masks to highlight spurious or core visual
features. Using this methodology, we introduce the {\it Salient Imagenet}
dataset containing core and spurious masks for a large set of samples from
Imagenet. Using this dataset, we show that several popular Imagenet models rely
heavily on various spurious features in their predictions, indicating the
standard accuracy alone is not sufficient to fully assess model performance.
Code and dataset for reproducing all experiments in the paper is available at
\url{https://github.com/singlasahil14/salient_imagenet}.
