Analyzing the Variance of Policy Gradient Estimators for the
  Linear-Quadratic Regulator
  We study the variance of the REINFORCE policy gradient estimator in
environments with continuous state and action spaces, linear dynamics,
quadratic cost, and Gaussian noise. These simple environments allow us to
derive bounds on the estimator variance in terms of the environment and noise
parameters. We compare the predictions of our bounds to the empirical variance
in simulation experiments.
