Scribbler: Controlling Deep Image Synthesis with Sketch and Color
  Recently, there have been several promising methods to generate realistic
imagery from deep convolutional networks. These methods sidestep the
traditional computer graphics rendering pipeline and instead generate imagery
at the pixel level by learning from large collections of photos (e.g. faces or
bedrooms). However, these methods are of limited utility because it is
difficult for a user to control what the network produces. In this paper, we
propose a deep adversarial image synthesis architecture that is conditioned on
sketched boundaries and sparse color strokes to generate realistic cars,
bedrooms, or faces. We demonstrate a sketch based image synthesis system which
allows users to 'scribble' over the sketch to indicate preferred color for
objects. Our network can then generate convincing images that satisfy both the
color and the sketch constraints of user. The network is feed-forward which
allows users to see the effect of their edits in real time. We compare to
recent work on sketch to image synthesis and show that our approach can
generate more realistic, more diverse, and more controllable outputs. The
architecture is also effective at user-guided colorization of grayscale images.
