Minimum Stein Discrepancy Estimators
  When maximum likelihood estimation is infeasible, one often turns to score
matching, contrastive divergence, or minimum probability flow to obtain
tractable parameter estimates. We provide a unifying perspective of these
techniques as minimum Stein discrepancy estimators, and use this lens to design
new diffusion kernel Stein discrepancy (DKSD) and diffusion score matching
(DSM) estimators with complementary strengths. We establish the consistency,
asymptotic normality, and robustness of DKSD and DSM estimators, then derive
stochastic Riemannian gradient descent algorithms for their efficient
optimisation. The main strength of our methodology is its flexibility, which
allows us to design estimators with desirable properties for specific models at
hand by carefully selecting a Stein discrepancy. We illustrate this advantage
for several challenging problems for score matching, such as non-smooth,
heavy-tailed or light-tailed densities.
