Hierarchical Mixtures of Generators for Adversarial Learning
  Generative adversarial networks (GANs) are deep neural networks that allow us
to sample from an arbitrary probability distribution without explicitly
estimating the distribution. There is a generator that takes a latent vector as
input and transforms it into a valid sample from the distribution. There is
also a discriminator that is trained to discriminate such fake samples from
true samples of the distribution; at the same time, the generator is trained to
generate fakes that the discriminator cannot tell apart from the true samples.
Instead of learning a global generator, a recent approach involves training
multiple generators each responsible from one part of the distribution. In this
work, we review such approaches and propose the hierarchical mixture of
generators, inspired from the hierarchical mixture of experts model, that
learns a tree structure implementing a hierarchical clustering with soft splits
in the decision nodes and local generators in the leaves. Since the generators
are combined softly, the whole model is continuous and can be trained using
gradient-based optimization, just like the original GAN model. Our experiments
on five image data sets, namely, MNIST, FashionMNIST, UTZap50K, Oxford Flowers,
and CelebA, show that our proposed model generates samples of high quality and
diversity in terms of popular GAN evaluation metrics. The learned hierarchical
structure also leads to knowledge extraction.
