Learning to Complete Code with Sketches
  Code completion is usually cast as a language modelling problem, i.e.,
continuing an input in a left-to-right fashion. However, in practice, some
parts of the completion (e.g., string literals) may be very hard to predict,
whereas subsequent parts directly follow from the context. To handle this, we
instead consider the scenario of generating code completions with "holes"
inserted in places where a model is uncertain. We develop Grammformer, a
Transformer-based model that guides code generation by the programming language
grammar, and compare it to a variety of more standard sequence models.
  We train the models on code completion for C# and Python given partial code
context. To evaluate models, we consider both ROUGE as well as a new metric
RegexAcc that measures success of generating completions matching long outputs
with as few holes as possible. In our experiments, Grammformer generates 10-50%
more accurate completions compared to traditional generative models and 37-50%
longer sketches compared to sketch-generating baselines trained with similar
techniques.
