Discovering Influential Factors in Variational Autoencoder
  In the field of machine learning, it is still a critical issue to identify
and supervise the learned representation without manually intervening or
intuition assistance to extract useful knowledge or serve for the downstream
tasks. In this work, we focus on supervising the influential factors extracted
by the variational autoencoder(VAE). The VAE is proposed to learn independent
low dimension representation while facing the problem that sometimes pre-set
factors are ignored. We argue that the mutual information of the input and each
learned factor of the representation plays a necessary indicator of discovering
the influential factors. We find the VAE objective inclines to induce mutual
information sparsity in factor dimension over the data intrinsic dimension and
results in some non-influential factors whose function on data reconstruction
could be ignored. We show mutual information also influences the lower bound of
VAE's reconstruction error and downstream classification task. To make such
indicator applicable, we design an algorithm for calculating the mutual
information for VAE and prove its consistency. Experimental results on MNIST,
CelebA and DEAP datasets show that mutual information can help determine
influential factors, of which some are interpretable and can be used to further
generation and classification tasks, and help discover the variant that
connects with emotion on DEAP dataset.
