RDP-GAN: A R\'enyi-Differential Privacy based Generative Adversarial
  Network
  Generative adversarial network (GAN) has attracted increasing attention
recently owing to its impressive ability to generate realistic samples with
high privacy protection. Without directly interactive with training examples,
the generative model can be fully used to estimate the underlying distribution
of an original dataset while the discriminative model can examine the quality
of the generated samples by comparing the label values with the training
examples. However, when GANs are applied on sensitive or private training
examples, such as medical or financial records, it is still probable to divulge
individuals' sensitive and private information. To mitigate this information
leakage and construct a private GAN, in this work we propose a
R\'enyi-differentially private-GAN (RDP-GAN), which achieves differential
privacy (DP) in a GAN by carefully adding random noises on the value of the
loss function during training. Moreover, we derive the analytical results of
the total privacy loss under the subsampling method and cumulated iterations,
which show its effectiveness on the privacy budget allocation. In addition, in
order to mitigate the negative impact brought by the injecting noise, we
enhance the proposed algorithm by adding an adaptive noise tuning step, which
will change the volume of added noise according to the testing accuracy.
Through extensive experimental results, we verify that the proposed algorithm
can achieve a better privacy level while producing high-quality samples
compared with a benchmark DP-GAN scheme based on noise perturbation on training
gradients.
