Factorized Gaussian Process Variational Autoencoders
  Variational autoencoders often assume isotropic Gaussian priors and
mean-field posteriors, hence do not exploit structure in scenarios where we may
expect similarity or consistency across latent variables. Gaussian process
variational autoencoders alleviate this problem through the use of a latent
Gaussian process, but lead to a cubic inference time complexity. We propose a
more scalable extension of these models by leveraging the independence of the
auxiliary features, which is present in many datasets. Our model factorizes the
latent kernel across these features in different dimensions, leading to a
significant speed-up (in theory and practice), while empirically performing
comparably to existing non-scalable approaches. Moreover, our approach allows
for additional modeling of global latent information and for more general
extrapolation to unseen input combinations.
