Banded Matrix Operators for Gaussian Markov Models in the Automatic
  Differentiation Era
  Banded matrices can be used as precision matrices in several models including
linear state-space models, some Gaussian processes, and Gaussian Markov random
fields. The aim of the paper is to make modern inference methods (such as
variational inference or gradient-based sampling) available for Gaussian models
with banded precision. We show that this can efficiently be achieved by
equipping an automatic differentiation framework, such as TensorFlow or
PyTorch, with some linear algebra operators dedicated to banded matrices. This
paper studies the algorithmic aspects of the required operators, details their
reverse-mode derivatives, and show that their complexity is linear in the
number of observations.
