Neural Nets via Forward State Transformation and Backward Loss
  Transformation
  This article studies (multilayer perceptron) neural networks with an emphasis
on the transformations involved --- both forward and backward --- in order to
develop a semantical/logical perspective that is in line with standard program
semantics. The common two-pass neural network training algorithms make this
viewpoint particularly fitting. In the forward direction, neural networks act
as state transformers. In the reverse direction, however, neural networks
change losses of outputs to losses of inputs, thereby acting like a
(real-valued) predicate transformer. In this way, backpropagation is functorial
by construction, as shown earlier in recent other work. We illustrate this
perspective by training a simple instance of a neural network.
