A piece-wise constant approximation for non-conjugate Gaussian Process
  models
  Gaussian Processes (GPs) are a versatile and popular method in Bayesian
Machine Learning. A common modification are Sparse Variational Gaussian
Processes (SVGPs) which are well suited to deal with large datasets. While GPs
allow to elegantly deal with Gaussian-distributed target variables in closed
form, their applicability can be extended to non-Gaussian data as well. These
extensions are usually impossible to treat in closed form and hence require
approximate solutions. This paper proposes to approximate the inverse-link
function, which is necessary when working with non-Gaussian likelihoods, by a
piece-wise constant function. It will be shown that this yields a closed form
solution for the corresponding SVGP lower bound. In addition, it is
demonstrated how the piece-wise constant function itself can be optimized,
resulting in an inverse-link function that can be learnt from the data at hand.
