ObjectNet Dataset: Reanalysis and Correction
  Recently, Barbu et al introduced a dataset called ObjectNet which includes
objects in daily life situations. They showed a dramatic performance drop of
the state of the art object recognition models on this dataset. Due to the
importance and implications of their results regarding generalization ability
of deep models, we take a second look at their findings. We highlight a major
problem with their work which is applying object recognizers to the scenes
containing multiple objects rather than isolated objects. The latter results in
around 20-30% performance gain using our code. Compared with the results
reported in the ObjectNet paper, we observe that around 10-15 % of the
performance loss can be recovered, without any test time data augmentation. In
accordance with Barbu et al.'s conclusions, however, we also conclude that deep
models suffer drastically on this dataset. Thus, we believe that ObjectNet
remains a challenging dataset for testing the generalization power of models
beyond datasets on which they have been trained.
