Approachability in Stackelberg Stochastic Games with Vector Costs
  The notion of approachability was introduced by Blackwell [1] in the context
of vector-valued repeated games. The famous Blackwell's approachability theorem
prescribes a strategy for approachability, i.e., for `steering' the average
cost of a given agent towards a given target set, irrespective of the
strategies of the other agents. In this paper, motivated by the multi-objective
optimization/decision making problems in dynamically changing environments, we
address the approachability problem in Stackelberg stochastic games with vector
valued cost functions. We make two main contributions. Firstly, we give a
simple and computationally tractable strategy for approachability for
Stackelberg stochastic games along the lines of Blackwell's. Secondly, we give
a reinforcement learning algorithm for learning the approachable strategy when
the transition kernel is unknown. We also recover as a by-product Blackwell's
necessary and sufficient condition for approachability for convex sets in this
set up and thus a complete characterization. We also give sufficient conditions
for non-convex sets.
