SCALOR: Generative World Models with Scalable Object Representations
  Scalability in terms of object density in a scene is a primary challenge in
unsupervised sequential object-oriented representation learning. Most of the
previous models have been shown to work only on scenes with a few objects. In
this paper, we propose SCALOR, a probabilistic generative world model for
learning SCALable Object-oriented Representation of a video. With the proposed
spatially-parallel attention and proposal-rejection mechanisms, SCALOR can deal
with orders of magnitude larger numbers of objects compared to the previous
state-of-the-art models. Additionally, we introduce a background module that
allows SCALOR to model complex dynamic backgrounds as well as many foreground
objects in the scene. We demonstrate that SCALOR can deal with crowded scenes
containing up to a hundred objects while jointly modeling complex dynamic
backgrounds. Importantly, SCALOR is the first unsupervised object
representation model shown to work for natural scenes containing several tens
of moving objects.
