Evaluating Disentanglement of Structured Representations
  We introduce the first metric for evaluating disentanglement at individual
hierarchy levels of a structured latent representation. Applied to
object-centric generative models, this offers a systematic, unified approach to
evaluating (i) object separation between latent slots (ii) disentanglement of
object properties inside individual slots (iii) disentanglement of intrinsic
and extrinsic object properties. We theoretically show that for structured
representations, our framework gives stronger guarantees of selecting a good
model than previous disentanglement metrics. Experimentally, we demonstrate
that viewing object compositionality as a disentanglement problem addresses
several issues with prior visual metrics of object separation. As a core
technical component, we present the first representation probing algorithm
handling slot permutation invariance.
