Group-disentangled Representation Learning with Weakly-Supervised
  Regularization
  Learning interpretable and human-controllable representations that uncover
factors of variation in data remains an ongoing key challenge in representation
learning. We investigate learning group-disentangled representations for groups
of factors with weak supervision. Existing techniques to address this challenge
merely constrain the approximate posterior by averaging over observations of a
shared group. As a result, observations with a common set of variations are
encoded to distinct latent representations, reducing their capacity to
disentangle and generalize to downstream tasks. In contrast to previous works,
we propose GroupVAE, a simple yet effective Kullback-Leibler (KL)
divergence-based regularization across shared latent representations to enforce
consistent and disentangled representations. We conduct a thorough evaluation
and demonstrate that our GroupVAE significantly improves group disentanglement.
Further, we demonstrate that learning group-disentangled representations
improve upon downstream tasks, including fair classification and 3D
shape-related tasks such as reconstruction, classification, and transfer
learning, and is competitive to supervised methods.
