On the Units of GANs (Extended Abstract)
  Generative Adversarial Networks (GANs) have achieved impressive results for
many real-world applications. As an active research topic, many GAN variants
have emerged with improvements in sample quality and training stability.
However, visualization and understanding of GANs is largely missing. How does a
GAN represent our visual world internally? What causes the artifacts in GAN
results? How do architectural choices affect GAN learning? Answering such
questions could enable us to develop new insights and better models. In this
work, we present an analytic framework to visualize and understand GANs at the
unit-, object-, and scene-level. We first identify a group of interpretable
units that are closely related to concepts with a segmentation-based network
dissection method. We quantify the causal effect of interpretable units by
measuring the ability of interventions to control objects in the output.
Finally, we examine the contextual relationship between these units and their
surrounding by inserting the discovered concepts into new images. We show
several practical applications enabled by our framework, from comparing
internal representations across different layers, models, and datasets, to
improving GANs by locating and removing artifact-causing units, to
interactively manipulating objects in the scene. We will open source our
interactive tools to help researchers and practitioners better understand their
models.
