Invert and Defend: Model-based Approximate Inversion of Generative
  Adversarial Networks for Secure Inference
  Inferring the latent variable generating a given test sample is a challenging
problem in Generative Adversarial Networks (GANs). In this paper, we propose
InvGAN - a novel framework for solving the inference problem in GANs, which
involves training an encoder network capable of inverting a pre-trained
generator network without access to any training data. Under mild assumptions,
we theoretically show that using InvGAN, we can approximately invert the
generations of any latent code of a trained GAN model. Furthermore, we
empirically demonstrate the superiority of our inference scheme by quantitative
and qualitative comparisons with other methods that perform a similar task. We
also show the effectiveness of our framework in the problem of adversarial
defenses where InvGAN can successfully be used as a projection-based defense
mechanism. Additionally, we show how InvGAN can be used to implement
reparameterization white-box attacks on projection-based defense mechanisms.
Experimental validation on several benchmark datasets demonstrate the efficacy
of our method in achieving improved performance on several white-box and
black-box attacks. Our code is available at
https://github.com/yogeshbalaji/InvGAN.
