Slimmable Generative Adversarial Networks
  Generative adversarial networks (GANs) have achieved remarkable progress in
recent years, but the continuously growing scale of models makes them
challenging to deploy widely in practical applications. In particular, for
real-time generation tasks, different devices require generators of different
sizes due to varying computing power. In this paper, we introduce slimmable
GANs (SlimGANs), which can flexibly switch the width of the generator to
accommodate various quality-efficiency trade-offs at runtime. Specifically, we
leverage multiple discriminators that share partial parameters to train the
slimmable generator. To facilitate the \textit{consistency} between generators
of different widths, we present a stepwise inplace distillation technique that
encourages narrow generators to learn from wide ones. As for class-conditional
generation, we propose a sliceable conditional batch normalization that
incorporates the label information into different widths. Our methods are
validated, both quantitatively and qualitatively, by extensive experiments and
a detailed ablation study.
