Learning to Compose Visual Relations
  The visual world around us can be described as a structured set of objects
and their associated relations. An image of a room may be conjured given only
the description of the underlying objects and their associated relations. While
there has been significant work on designing deep neural networks which may
compose individual objects together, less work has been done on composing the
individual relations between objects. A principal difficulty is that while the
placement of objects is mutually independent, their relations are entangled and
dependent on each other. To circumvent this issue, existing works primarily
compose relations by utilizing a holistic encoder, in the form of text or
graphs. In this work, we instead propose to represent each relation as an
unnormalized density (an energy-based model), enabling us to compose separate
relations in a factorized manner. We show that such a factorized decomposition
allows the model to both generate and edit scenes that have multiple sets of
relations more faithfully. We further show that decomposition enables our model
to effectively understand the underlying relational scene structure. Project
page at: https://composevisualrelations.github.io/.
