Unsupervised Learning of Compositional Energy Concepts
  Humans are able to rapidly understand scenes by utilizing concepts extracted
from prior experience. Such concepts are diverse, and include global scene
descriptors, such as the weather or lighting, as well as local scene
descriptors, such as the color or size of a particular object. So far,
unsupervised discovery of concepts has focused on either modeling the global
scene-level or the local object-level factors of variation, but not both. In
this work, we propose COMET, which discovers and represents concepts as
separate energy functions, enabling us to represent both global concepts as
well as objects under a unified framework. COMET discovers energy functions
through recomposing the input image, which we find captures independent factors
without additional supervision. Sample generation in COMET is formulated as an
optimization process on underlying energy functions, enabling us to generate
images with permuted and composed concepts. Finally, discovered visual concepts
in COMET generalize well, enabling us to compose concepts between separate
modalities of images as well as with other concepts discovered by a separate
instance of COMET trained on a different dataset. Code and data available at
https://energy-based-model.github.io/comet/.
