Faster Game Solving via Predictive Blackwell Approachability: Connecting
  Regret Matching and Mirror Descent
  Blackwell approachability is a framework for reasoning about repeated games
with vector-valued payoffs. We introduce predictive Blackwell approachability,
where an estimate of the next payoff vector is given, and the decision maker
tries to achieve better performance based on the accuracy of that estimator. In
order to derive algorithms that achieve predictive Blackwell approachability,
we start by showing a powerful connection between four well-known algorithms.
Follow-the-regularized-leader (FTRL) and online mirror descent (OMD) are the
most prevalent regret minimizers in online convex optimization. In spite of
this prevalence, the regret matching (RM) and regret matching+ (RM+) algorithms
have been preferred in the practice of solving large-scale games (as the local
regret minimizers within the counterfactual regret minimization framework). We
show that RM and RM+ are the algorithms that result from running FTRL and OMD,
respectively, to select the halfspace to force at all times in the underlying
Blackwell approachability game. By applying the predictive variants of FTRL or
OMD to this connection, we obtain predictive Blackwell approachability
algorithms, as well as predictive variants of RM and RM+. In experiments across
18 common zero-sum extensive-form benchmark games, we show that predictive RM+
coupled with counterfactual regret minimization converges vastly faster than
the fastest prior algorithms (CFR+, DCFR, LCFR) across all games but two of the
poker games, sometimes by two or more orders of magnitude.
