Creating User Interface Mock-ups from High-Level Text Descriptions with
  Deep-Learning Models
  The design process of user interfaces (UIs) often begins with articulating
high-level design goals. Translating these high-level design goals into
concrete design mock-ups, however, requires extensive effort and UI design
expertise. To facilitate this process for app designers and developers, we
introduce three deep-learning techniques to create low-fidelity UI mock-ups
from a natural language phrase that describes the high-level design goal (e.g.
"pop up displaying an image and other options"). In particular, we contribute
two retrieval-based methods and one generative method, as well as
pre-processing and post-processing techniques to ensure the quality of the
created UI mock-ups. We quantitatively and qualitatively compare and contrast
each method's ability in suggesting coherent, diverse and relevant UI design
mock-ups. We further evaluate these methods with 15 professional UI designers
and practitioners to understand each method's advantages and disadvantages. The
designers responded positively to the potential of these methods for assisting
the design process.
