A Simple Differentiable Programming Language
  Automatic differentiation plays a prominent role in scientific computing and
in modern machine learning, often in the context of powerful programming
systems. The relation of the various embodiments of automatic differentiation
to the mathematical notion of derivative is not always entirely
clear---discrepancies can arise, sometimes inadvertently. In order to study
automatic differentiation in such programming contexts, we define a small but
expressive programming language that includes a construct for reverse-mode
differentiation. We give operational and denotational semantics for this
language. The operational semantics employs popular implementation techniques,
while the denotational semantics employs notions of differentiation familiar
from real analysis. We establish that these semantics coincide.
