Is Supervised Learning With Adversarial Features Provably Better Than
  Sole Supervision?
  Generative Adversarial Networks (GAN) have shown promising results on a wide
variety of complex tasks. Recent experiments show adversarial training provides
useful gradients to the generator that helps attain better performance. In this
paper, we intend to theoretically analyze whether supervised learning with
adversarial features can outperform sole supervision, or not. First, we show
that supervised learning without adversarial features suffer from vanishing
gradient issue in near optimal region. Second, we analyze how adversarial
learning augmented with supervised signal mitigates this vanishing gradient
issue. Finally, we prove our main result that shows supervised learning with
adversarial features can be better than sole supervision (under some mild
assumptions). We support our main result on two fronts (i) expected empirical
risk and (ii) rate of convergence.
