Functional Tensors for Probabilistic Programming
  It is a significant challenge to design probabilistic programming systems
that can accommodate a wide variety of inference strategies within a unified
framework. Noting that the versatility of modern automatic differentiation
frameworks is based in large part on the unifying concept of tensors, we
describe a software abstraction for integration --functional tensors-- that
captures many of the benefits of tensors, while also being able to describe
continuous probability distributions. Moreover, functional tensors are a
natural candidate for generalized variable elimination and parallel-scan
filtering algorithms that enable parallel exact inference for a large family of
tractable modeling motifs. We demonstrate the versatility of functional tensors
by integrating them into the modeling frontend and inference backend of the
Pyro programming language. In experiments we show that the resulting framework
enables a large variety of inference strategies, including those that mix exact
and approximate inference.
