Memorization and Generalization in Neural Code Intelligence Models
  Deep Neural Networks are capable of learning highly generalizable patterns
from large datasets of source code through millions of parameters. This large
capacity also renders them prone to memorizing data points. Recent work
suggests that the memorization risk manifests especially strongly when the
training dataset is noisy, involving many ambiguous or questionable samples,
and memorization is the only recourse. Unfortunately, most code intelligence
tasks rely on rather noise-prone and repetitive data sources, such as code from
GitHub. Given the sheer size of such corpora, determining the role and extent
of noise in these is beyond manual inspection. In this paper, we propose an
alternative analysis: we evaluate the impact of the noise on training neural
models of source code by introducing targeted noise to the dataset of several
state-of-the-art neural intelligence models and benchmarks based on Java and
Python codebases. By studying the resulting behavioral changes at various rates
of noise, and across a wide range of metrics, we can characterize both typical
generalizing and problematic memorization-like learning of models of source
code. Our results highlight important risks: millions of trainable parameters
allow the neural networks to memorize anything, including noisy data, and
provide a false sense of generalization. At the same time, the metrics used to
analyze this phenomenon proved surprisingly useful for detecting and
quantifying such effects, offering a powerful toolset for creating reliable
models of code.
