Rayleigh-Gauss-Newton optimization with enhanced sampling for
  variational Monte Carlo
  Variational Monte Carlo (VMC) is an approach for computing ground-state
wavefunctions that has recently become more powerful due to the introduction of
neural network-based wavefunction parametrizations. However, efficiently
training neural wavefunctions to converge to an energy minimum remains a
difficult problem. In this work, we analyze optimization and sampling methods
used in VMC and introduce alterations to improve their performance. First,
based on theoretical convergence analysis in a noiseless setting, we motivate a
new optimizer that we call the Rayleigh-Gauss-Newton method, which can improve
upon gradient descent and natural gradient descent to achieve superlinear
convergence with little added computational cost. Second, in order to realize
this favorable comparison in the presence of stochastic noise, we analyze the
effect of sampling error on VMC parameter updates and experimentally
demonstrate that it can be reduced by the parallel tempering method. In
particular, we demonstrate that RGN can be made robust to energy spikes that
occur when new regions of configuration space become available to the sampler
over the course of optimization. Finally, putting theory into practice, we
apply our enhanced optimization and sampling methods to the transverse-field
Ising and XXZ models on large lattices, yielding ground-state energy estimates
with remarkably high accuracy after just 200-500 parameter updates.
