Policy Gradient for Coherent Risk Measures
  Several authors have recently developed risk-sensitive policy gradient
methods that augment the standard expected cost minimization problem with a
measure of variability in cost. These studies have focused on specific
risk-measures, such as the variance or conditional value at risk (CVaR). In
this work, we extend the policy gradient method to the whole class of coherent
risk measures, which is widely accepted in finance and operations research,
among other fields. We consider both static and time-consistent dynamic risk
measures. For static risk measures, our approach is in the spirit of policy
gradient algorithms and combines a standard sampling approach with convex
programming. For dynamic risk measures, our approach is actor-critic style and
involves explicit approximation of value function. Most importantly, our
contribution presents a unified approach to risk-sensitive reinforcement
learning that generalizes and extends previous results.
