CoT: Cooperative Training for Generative Modeling of Discrete Data
  In this paper, we study the generative models of sequential discrete data. To
tackle the exposure bias problem inherent in maximum likelihood estimation
(MLE), generative adversarial networks (GANs) are introduced to penalize the
unrealistic generated samples. To exploit the supervision signal from the
discriminator, most previous models leverage REINFORCE to address the
non-differentiable problem of sequential discrete data. However, because of the
unstable property of the training signal during the dynamic process of
adversarial training, the effectiveness of REINFORCE, in this case, is hardly
guaranteed. To deal with such a problem, we propose a novel approach called
Cooperative Training (CoT) to improve the training of sequence generative
models. CoT transforms the min-max game of GANs into a joint maximization
framework and manages to explicitly estimate and optimize Jensen-Shannon
divergence. Moreover, CoT works without the necessity of pre-training via MLE,
which is crucial to the success of previous methods. In the experiments,
compared to existing state-of-the-art methods, CoT shows superior or at least
competitive performance on sample quality, diversity, as well as training
stability.
