Scones: Towards Conversational Authoring of Sketches
  Iteratively refining and critiquing sketches are crucial steps to developing
effective designs. We introduce Scones, a mixed-initiative,
machine-learning-driven system that enables users to iteratively author
sketches from text instructions. Scones is a novel deep-learning-based system
that iteratively generates scenes of sketched objects composed with semantic
specifications from natural language. Scones exceeds state-of-the-art
performance on a text-based scene modification task, and introduces a
mask-conditioned sketching model that can generate sketches with poses
specified by high-level scene information. In an exploratory user evaluation of
Scones, participants reported enjoying an iterative drawing task with Scones,
and suggested additional features for further applications. We believe Scones
is an early step towards automated, intelligent systems that support
human-in-the-loop applications for communicating ideas through sketching in art
and design.
