Convergence of Learning Dynamics in Stackelberg Games
  This paper investigates the convergence of learning dynamics in Stackelberg
games. In the class of games we consider, there is a hierarchical game being
played between a leader and a follower with continuous action spaces. We
establish a number of connections between the Nash and Stackelberg equilibrium
concepts and characterize conditions under which attracting critical points of
simultaneous gradient descent are Stackelberg equilibria in zero-sum games.
Moreover, we show that the only stable critical points of the Stackelberg
gradient dynamics are Stackelberg equilibria in zero-sum games. Using this
insight, we develop a gradient-based update for the leader while the follower
employs a best response strategy for which each stable critical point is
guaranteed to be a Stackelberg equilibrium in zero-sum games. As a result, the
learning rule provably converges to a Stackelberg equilibria given an
initialization in the region of attraction of a stable critical point. We then
consider a follower employing a gradient-play update rule instead of a best
response strategy and propose a two-timescale algorithm with similar asymptotic
convergence guarantees. For this algorithm, we also provide finite-time high
probability bounds for local convergence to a neighborhood of a stable
Stackelberg equilibrium in general-sum games. Finally, we present extensive
numerical results that validate our theory, provide insights into the
optimization landscape of generative adversarial networks, and demonstrate that
the learning dynamics we propose can effectively train generative adversarial
networks.
