GANs with Conditional Independence Graphs: On Subadditivity of
  Probability Divergences
  Generative Adversarial Networks (GANs) are modern methods to learn the
underlying distribution of a data set. GANs have been widely used in sample
synthesis, de-noising, domain transfer, etc. GANs, however, are designed in a
model-free fashion where no additional information about the underlying
distribution is available. In many applications, however, practitioners have
access to the underlying independence graph of the variables, either as a
Bayesian network or a Markov Random Field (MRF). We ask: how can one use this
additional information in designing model-based GANs? In this paper, we provide
theoretical foundations to answer this question by studying subadditivity
properties of probability divergences, which establish upper bounds on the
distance between two high-dimensional distributions by the sum of distances
between their marginals over (local) neighborhoods of the graphical structure
of the Bayes-net or the MRF. We prove that several popular probability
divergences satisfy some notion of subadditivity under mild conditions. These
results lead to a principled design of a model-based GAN that uses a set of
simple discriminators on the neighborhoods of the Bayes-net/MRF, rather than a
giant discriminator on the entire network, providing significant statistical
and computational benefits. Our experiments on synthetic and real-world
datasets demonstrate the benefits of our principled design of model-based GANs.
