The Benefits of Pairwise Discriminators for Adversarial Training
  Adversarial training methods typically align distributions by solving
two-player games. However, in most current formulations, even if the generator
aligns perfectly with data, a sub-optimal discriminator can still drive the two
apart. Absent additional regularization, the instability can manifest itself as
a never-ending game. In this paper, we introduce a family of objectives by
leveraging pairwise discriminators, and show that only the generator needs to
converge. The alignment, if achieved, would be preserved with any
discriminator. We provide sufficient conditions for local convergence;
characterize the capacity balance that should guide the discriminator and
generator choices; and construct examples of minimally sufficient
discriminators. Empirically, we illustrate the theory and the effectiveness of
our approach on synthetic examples. Moreover, we show that practical methods
derived from our approach can better generate higher-resolution images.
