Distributional Advantage Actor-Critic
  In traditional reinforcement learning, an agent maximizes the reward
collected during its interaction with the environment by approximating the
optimal policy through the estimation of value functions. Typically, given a
state s and action a, the corresponding value is the expected discounted sum of
rewards. The optimal action is then chosen to be the action a with the largest
value estimated by value function. However, recent developments have shown both
theoretical and experimental evidence of superior performance when value
function is replaced with value distribution in context of deep Q learning [1].
In this paper, we develop a new algorithm that combines advantage actor-critic
with value distribution estimated by quantile regression. We evaluated this new
algorithm, termed Distributional Advantage Actor-Critic (DA2C or QR-A2C) on a
variety of tasks, and observed it to achieve at least as good as baseline
algorithms, and outperforming baseline in some tasks with smaller variance and
increased stability.
