Multiscale Generative Models: Improving Performance of a Generative
  Model Using Feedback from Other Dependent Generative Models
  Realistic fine-grained multi-agent simulation of real-world complex systems
is crucial for many downstream tasks such as reinforcement learning. Recent
work has used generative models (GANs in particular) for providing
high-fidelity simulation of real-world systems. However, such generative models
are often monolithic and miss out on modeling the interaction in multi-agent
systems. In this work, we take a first step towards building multiple
interacting generative models (GANs) that reflects the interaction in real
world. We build and analyze a hierarchical set-up where a higher-level GAN is
conditioned on the output of multiple lower-level GANs. We present a technique
of using feedback from the higher-level GAN to improve performance of
lower-level GANs. We mathematically characterize the conditions under which our
technique is impactful, including understanding the transfer learning nature of
our set-up. We present three distinct experiments on synthetic data, time
series data, and image domain, revealing the wide applicability of our
technique.
