A Small Gain Analysis of Single Timescale Actor Critic
  We consider a version of actor-critic which uses proportional step-sizes and
only one critic update with a single sample from the stationary distribution
per actor step. We provide an analysis of this method using the small-gain
theorem. Specifically, we prove that this method can be used to find a
stationary point, and that the resulting sample complexity improves the state
of the art for actor-critic methods to $O \left(\mu^{-2} \epsilon^{-2} \right)$
to find an $\epsilon$-approximate stationary point where $\mu$ is the condition
number associated with the critic.
