ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object
  Segmentation
  There has been a recent surge in methods that aim to decompose and segment
scenes into multiple objects in an unsupervised manner, i.e., unsupervised
multi-object segmentation. Performing such a task is a long-standing goal of
computer vision, offering to unlock object-level reasoning without requiring
dense annotations to train segmentation models. Despite significant progress,
current models are developed and trained on visually simple scenes depicting
mono-colored objects on plain backgrounds. The natural world, however, is
visually complex with confounding aspects such as diverse textures and
complicated lighting effects. In this study, we present a new benchmark called
ClevrTex, designed as the next challenge to compare, evaluate and analyze
algorithms. ClevrTex features synthetic scenes with diverse shapes, textures
and photo-mapped materials, created using physically based rendering
techniques. It includes 50k examples depicting 3-10 objects arranged on a
background, created using a catalog of 60 materials, and a further test set
featuring 10k images created using 25 different materials. We benchmark a large
set of recent unsupervised multi-object segmentation models on ClevrTex and
find all state-of-the-art approaches fail to learn good representations in the
textured setting, despite impressive performance on simpler data. We also
create variants of the ClevrTex dataset, controlling for different aspects of
scene complexity, and probe current approaches for individual shortcomings.
Dataset and code are available at
https://www.robots.ox.ac.uk/~vgg/research/clevrtex.
