Named Tensor Notation
  We propose a notation for tensors with named axes, which relieves the author,
reader, and future implementers from the burden of keeping track of the order
of axes and the purpose of each. It also makes it easy to extend operations on
low-order tensors to higher order ones (e.g., to extend an operation on images
to minibatches of images, or extend the attention mechanism to multiple
attention heads). After a brief overview of our notation, we illustrate it
through several examples from modern machine learning, from building blocks
like attention and convolution to full models like Transformers and LeNet.
Finally, we give formal definitions and describe some extensions. Our proposals
build on ideas from many previous papers and software libraries. We hope that
this document will encourage more authors to use named tensors, resulting in
clearer papers and less bug-prone implementations.
  The source code for this document can be found at
https://github.com/namedtensor/notation/. We invite anyone to make comments on
this proposal by submitting issues or pull requests on this repository.
