Transforming Gaussian Processes With Normalizing Flows
  Gaussian Processes (GPs) can be used as flexible, non-parametric function
priors. Inspired by the growing body of work on Normalizing Flows, we enlarge
this class of priors through a parametric invertible transformation that can be
made input-dependent. Doing so also allows us to encode interpretable prior
knowledge (e.g., boundedness constraints). We derive a variational
approximation to the resulting Bayesian inference problem, which is as fast as
stochastic variational GP regression (Hensman et al., 2013; Dezfouli and
Bonilla,2015). This makes the model a computationally efficient alternative to
other hierarchical extensions of GP priors (Lazaro-Gredilla,2012; Damianou and
Lawrence, 2013). The resulting algorithm's computational and inferential
performance is excellent, and we demonstrate this on a range of data sets. For
example, even with only 5 inducing points and an input-dependent flow, our
method is consistently competitive with a standard sparse GP fitted using 100
inducing points.
