New Tricks for Estimating Gradients of Expectations
  We introduce a family of pairwise stochastic gradient estimators for
gradients of expectations, which are related to the log-derivative trick, but
involve pairwise interactions between samples. The simplest example of our new
estimator, dubbed the fundamental trick estimator, is shown to arise from
either a) introducing and approximating an integral representation based on the
fundamental theorem of calculus, or b) applying the reparameterisation trick to
an implicit parameterisation under infinitesimal perturbation of the
parameters. From the former perspective we generalise to a reproducing kernel
Hilbert space representation, giving rise to a locality parameter in the
pairwise interactions mentioned above, yielding our representer trick
estimator. The resulting estimators are unbiased and shown to offer an
independent component of useful information in comparison with the
log-derivative estimator. We provide a further novel theoretical analysis which
further characterises the variance reduction afforded by the new techniques.
Promising analytical and numerical examples confirm the theory and intuitions
behind the new estimators.
