HumanACGAN: conditional generative adversarial network with human-based
  auxiliary classifier and its evaluation in phoneme perception
  We propose a conditional generative adversarial network (GAN) incorporating
humans' perceptual evaluations. A deep neural network (DNN)-based generator of
a GAN can represent a real-data distribution accurately but can never represent
a human-acceptable distribution, which are ranges of data in which humans
accept the naturalness regardless of whether the data are real or not. A
HumanGAN was proposed to model the human-acceptable distribution. A DNN-based
generator is trained using a human-based discriminator, i.e., humans'
perceptual evaluations, instead of the GAN's DNN-based discriminator. However,
the HumanGAN cannot represent conditional distributions. This paper proposes
the HumanACGAN, a theoretical extension of the HumanGAN, to deal with
conditional human-acceptable distributions. Our HumanACGAN trains a DNN-based
conditional generator by regarding humans as not only a discriminator but also
an auxiliary classifier. The generator is trained by deceiving the human-based
discriminator that scores the unconditioned naturalness and the human-based
classifier that scores the class-conditioned perceptual acceptability. The
training can be executed using the backpropagation algorithm involving humans'
perceptual evaluations. Our experimental results in phoneme perception
demonstrate that our HumanACGAN can successfully train this conditional
generator.
