Quantile-Based Policy Optimization for Reinforcement Learning
  Classical reinforcement learning (RL) aims to optimize the expected
cumulative rewards. In this work, we consider the RL setting where the goal is
to optimize the quantile of the cumulative rewards. We parameterize the policy
controlling actions by neural networks and propose a novel policy gradient
algorithm called Quantile-Based Policy Optimization (QPO) and its variant
Quantile-Based Proximal Policy Optimization (QPPO) to solve deep RL problems
with quantile objectives. QPO uses two coupled iterations running at different
time scales for simultaneously estimating quantiles and policy parameters and
is shown to converge to the global optimal policy under certain conditions. Our
numerical results demonstrate that the proposed algorithms outperform the
existing baseline algorithms under the quantile criterion.
