Non-Asymptotic Error Bounds for Bidirectional GANs
  We derive nearly sharp bounds for the bidirectional GAN (BiGAN) estimation
error under the Dudley distance between the latent joint distribution and the
data joint distribution with appropriately specified architecture of the neural
networks used in the model. To the best of our knowledge, this is the first
theoretical guarantee for the bidirectional GAN learning approach. An appealing
feature of our results is that they do not assume the reference and the data
distributions to have the same dimensions or these distributions to have
bounded support. These assumptions are commonly assumed in the existing
convergence analysis of the unidirectional GANs but may not be satisfied in
practice. Our results are also applicable to the Wasserstein bidirectional GAN
if the target distribution is assumed to have a bounded support. To prove these
results, we construct neural network functions that push forward an empirical
distribution to another arbitrary empirical distribution on a possibly
different-dimensional space. We also develop a novel decomposition of the
integral probability metric for the error analysis of bidirectional GANs. These
basic theoretical results are of independent interest and can be applied to
other related learning problems.
