Is Disentanglement enough? On Latent Representations for Controllable
  Music Generation
  Improving controllability or the ability to manipulate one or more attributes
of the generated data has become a topic of interest in the context of deep
generative models of music. Recent attempts in this direction have relied on
learning disentangled representations from data such that the underlying
factors of variation are well separated. In this paper, we focus on the
relationship between disentanglement and controllability by conducting a
systematic study using different supervised disentanglement learning algorithms
based on the Variational Auto-Encoder (VAE) architecture. Our experiments show
that a high degree of disentanglement can be achieved by using different forms
of supervision to train a strong discriminative encoder. However, in the
absence of a strong generative decoder, disentanglement does not necessarily
imply controllability. The structure of the latent space with respect to the
VAE-decoder plays an important role in boosting the ability of a generative
model to manipulate different attributes. To this end, we also propose methods
and metrics to help evaluate the quality of a latent space with respect to the
afforded degree of controllability.
