Using Constant Learning Rate of Two Time-Scale Update Rule for Training
  Generative Adversarial Networks
  Previous numerical results have shown that a two time-scale update rule
(TTUR) using constant learning rates is practically useful for training
generative adversarial networks (GANs). Meanwhile, a theoretical analysis of
TTUR to find a stationary local Nash equilibrium of a Nash equilibrium problem
with two players, a discriminator and a generator, has been given using
decaying learning rates. In this paper, we give a theoretical analysis of TTUR
using constant learning rates to bridge the gap between theory and practice. In
particular, we show that, for TTUR using constant learning rates, the number of
steps needed to find a stationary local Nash equilibrium decreases as the batch
size increases. We also provide numerical results to support our theoretical
analyzes.
