Deep Energy Estimator Networks
  Density estimation is a fundamental problem in statistical learning. This
problem is especially challenging for complex high-dimensional data due to the
curse of dimensionality. A promising solution to this problem is given here in
an inference-free hierarchical framework that is built on score matching. We
revisit the Bayesian interpretation of the score function and the Parzen score
matching, and construct a multilayer perceptron with a scalable objective for
learning the energy (i.e. the unnormalized log-density), which is then
optimized with stochastic gradient descent. In addition, the resulting deep
energy estimator network (DEEN) is designed as products of experts. We present
the utility of DEEN in learning the energy, the score function, and in
single-step denoising experiments for synthetic and high-dimensional data. We
also diagnose stability problems in the direct estimation of the score function
that had been observed for denoising autoencoders.
