OpenHoldem: A Benchmark for Large-Scale Imperfect-Information Game
  Research
  Owning to the unremitting efforts by a few institutes, significant progress
has recently been made in designing superhuman AIs in No-limit Texas Hold'em
(NLTH), the primary testbed for large-scale imperfect-information game
research. However, it remains challenging for new researchers to study this
problem since there are no standard benchmarks for comparing with existing
methods, which seriously hinders further developments in this research area. In
this work, we present OpenHoldem, an integrated toolkit for large-scale
imperfect-information game research using NLTH. OpenHoldem makes three main
contributions to this research direction: 1) a standardized evaluation protocol
for thoroughly evaluating different NLTH AIs, 2) four publicly available strong
baselines for NLTH AI, and 3) an online testing platform with easy-to-use APIs
for public NLTH AI evaluation. We have released OpenHoldem at holdem.ia.ac.cn,
hoping it facilitates further studies on the unsolved theoretical and
computational issues in this area and cultivate crucial research problems like
opponent modeling and human-computer interactive learning.
