Proximal Policy Optimization for Improved Convergence in IRGAN
  IRGAN is an information retrieval (IR) modeling approach that uses a
theoretical minimax game between a generative and a discriminative model to
iteratively optimize both of them, hence unifying the generative and
discriminative approaches. Despite significant performance improvements in
several information retrieval tasks, IRGAN training is an unstable process, and
the solution varies largely with the random parameter initialization. In this
work, we present an improved training objective based on proximal policy
optimization objective and Gumbel-Softmax based sampling for the generator. We
also propose a modified training algorithm which takes a single gradient update
on both the generator as well as discriminator for each iteration step. We
present empirical evidence of the improved convergence of the proposed model
over the original IRGAN and a comparison on three different IR tasks on
benchmark datasets is also discussed, emphasizing the proposed model's superior
performance.
