Reparameterization Gradient for Non-differentiable Models
  We present a new algorithm for stochastic variational inference that targets
at models with non-differentiable densities. One of the key challenges in
stochastic variational inference is to come up with a low-variance estimator of
the gradient of a variational objective. We tackle the challenge by
generalizing the reparameterization trick, one of the most effective techniques
for addressing the variance issue for differentiable models, so that the trick
works for non-differentiable models as well. Our algorithm splits the space of
latent variables into regions where the density of the variables is
differentiable, and their boundaries where the density may fail to be
differentiable. For each differentiable region, the algorithm applies the
standard reparameterization trick and estimates the gradient restricted to the
region. For each potentially non-differentiable boundary, it uses a form of
manifold sampling and computes the direction for variational parameters that,
if followed, would increase the boundary's contribution to the variational
objective. The sum of all the estimates becomes the gradient estimate of our
algorithm. Our estimator enjoys the reduced variance of the reparameterization
gradient while remaining unbiased even for non-differentiable models. The
experiments with our preliminary implementation confirm the benefit of reduced
variance and unbiasedness.
