Policy Gradient Methods for Reinforcement Learning with Function
  Approximation and Action-Dependent Baselines
  We show how an action-dependent baseline can be used by the policy gradient
theorem using function approximation, originally presented with
action-independent baselines by (Sutton et al. 2000).
