Collaborative Training of GANs in Continuous and Discrete Spaces for
  Text Generation
  Applying generative adversarial networks (GANs) to text-related tasks is
challenging due to the discrete nature of language. One line of research
resolves this issue by employing reinforcement learning (RL) and optimizing the
next-word sampling policy directly in a discrete action space. Such methods
compute the rewards from complete sentences and avoid error accumulation due to
exposure bias. Other approaches employ approximation techniques that map the
text to continuous representation in order to circumvent the non-differentiable
discrete process. Particularly, autoencoder-based methods effectively produce
robust representations that can model complex discrete structures. In this
paper, we propose a novel text GAN architecture that promotes the collaborative
training of the continuous-space and discrete-space methods. Our method employs
an autoencoder to learn an implicit data manifold, providing a learning
objective for adversarial training in a continuous space. Furthermore, the
complete textual output is directly evaluated and updated via RL in a discrete
space. The collaborative interplay between the two adversarial trainings
effectively regularize the text representations in different spaces. The
experimental results on three standard benchmark datasets show that our model
substantially outperforms state-of-the-art text GANs with respect to quality,
diversity, and global consistency.
