Response-Based Approachability and its Application to Generalized
  No-Regret Algorithms
  Approachability theory, introduced by Blackwell (1956), provides fundamental
results on repeated games with vector-valued payoffs, and has been usefully
applied since in the theory of learning in games and to learning algorithms in
the online adversarial setup. Given a repeated game with vector payoffs, a
target set $S$ is approachable by a certain player (the agent) if he can ensure
that the average payoff vector converges to that set no matter what his
adversary opponent does. Blackwell provided two equivalent sets of conditions
for a convex set to be approachable. The first (primary) condition is a
geometric separation condition, while the second (dual) condition requires that
the set be {\em non-excludable}, namely that for every mixed action of the
opponent there exists a mixed action of the agent (a {\em response}) such that
the resulting payoff vector belongs to $S$. Existing approachability algorithms
rely on the primal condition and essentially require to compute at each stage a
projection direction from a given point to $S$. In this paper, we introduce an
approachability algorithm that relies on Blackwell's {\em dual} condition.
Thus, rather than projection, the algorithm relies on computation of the
response to a certain action of the opponent at each stage. The utility of the
proposed algorithm is demonstrated by applying it to certain generalizations of
the classical regret minimization problem, which include regret minimization
with side constraints and regret minimization for global cost functions. In
these problems, computation of the required projections is generally complex
but a response is readily obtainable.
