Disentanglement Learning for Variational Autoencoders Applied to
  Audio-Visual Speech Enhancement
  Recently, the standard variational autoencoder has been successfully used to
learn a probabilistic prior over speech signals, which is then used to perform
speech enhancement. Variational autoencoders have then been conditioned on a
label describing a high-level speech attribute (e.g. speech activity) that
allows for a more explicit control of speech generation. However, the label is
not guaranteed to be disentangled from the other latent variables, which
results in limited performance improvements compared to the standard
variational autoencoder. In this work, we propose to use an adversarial
training scheme for variational autoencoders to disentangle the label from the
other latent variables. At training, we use a discriminator that competes with
the encoder of the variational autoencoder. Simultaneously, we also use an
additional encoder that estimates the label for the decoder of the variational
autoencoder, which proves to be crucial to learn disentanglement. We show the
benefit of the proposed disentanglement learning when a voice activity label,
estimated from visual data, is used for speech enhancement.
