Accelerated Reinforcement Learning
  Policy gradient methods are widely used in reinforcement learning algorithms
to search for better policies in the parameterized policy space. They do
gradient search in the policy space and are known to converge very slowly.
Nesterov developed an accelerated gradient search algorithm for convex
optimization problems. This has been recently extended for non-convex and also
stochastic optimization. We use Nesterov's acceleration for policy gradient
search in the well-known actor-critic algorithm and show the convergence using
ODE method. We tested this algorithm on a scheduling problem. Here an incoming
job is scheduled into one of the four queues based on the queue lengths. We see
from experimental results that algorithm using Nesterov's acceleration has
significantly better performance compared to algorithm which do not use
acceleration. To the best of our knowledge this is the first time Nesterov's
acceleration has been used with actor-critic algorithm.
