Static Prediction of Runtime Errors by Learning to Execute Programs with
  External Resource Descriptions
  The execution behavior of a program often depends on external resources, such
as program inputs or file contents, and so cannot be run in isolation.
Nevertheless, software developers benefit from fast iteration loops where
automated tools identify errors as early as possible, even before programs can
be compiled and run. This presents an interesting machine learning challenge:
can we predict runtime errors in a "static" setting, where program execution is
not possible? Here, we introduce a real-world dataset and task for predicting
runtime errors, which we show is difficult for generic models like
Transformers. We approach this task by developing an interpreter-inspired
architecture with an inductive bias towards mimicking program executions, which
models exception handling and "learns to execute" descriptions of the contents
of external resources. Surprisingly, we show that the model can also predict
the location of the error, despite being trained only on labels indicating the
presence/absence and kind of error. In total, we present a practical and
difficult-yet-approachable challenge problem related to learning program
execution and we demonstrate promising new capabilities of interpreter-inspired
machine learning models for code.
