{'fraction_non_alphanumeric': 0.02837997635001971, 'fraction_numerical': 0.008277493102089082, 'mean_word_length': 4.957746478873239, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Combining Monte-Carlo Tree Search and Opponent Modelling in Poker

Games have always been an area to receive much attention from researchers in artificial intelligence. During the last decennia, research was focused on games like chess, resulting in a computer player (Deep Blue) that was able to beat the world champion (Garri Kasparov). In the past years, researchers became increasingly interested in imperfect-information games like poker. In these domains, a different approach should be used to be able to develop a computer player that can compete against human professionals. The majority of previous research focuses on limit poker or on poker with only one opponent (heads-up), while the most popular poker variant nowadays is a no-limit variant with several players. This research introduces a new search technique for games in the domain of poker: Monte-Carlo Tree Search (MCTS). This technique has already proved valuable in the field of Go [11]. MCTS is able to deal with the larger state space associated with no-limit poker against multiple opponents. Furthermore, if a computer poker player should be able to compete against poker professionals, it should use some sort of opponent modelling [5]. Therefore, we wanted to show that the Bayesian approach to opponent modelling proposed by Ponsen et al. [16] could be a valuable component of a computer poker player. We tested both MCTS and Bayesian opponent modelling in 2 experiments: in the first experiment, our poker bot played against a relatively simple rulebased computer player (ACE1). In the second experiment, our computer poker player faced Poki, a more sophisticated computer player [5]. Results from the experiments show that: (1) MCTS is a valuable search technique which can yield good results against a relatively simple opponent. However, unless the technique improves, MCTS itself will not be able to compete against experienced human poker players. Our second conclusion is that (2) a poker AI incorporating the Bayesian approach to opponent modelling is more effective than a poker AI who does not use this form of opponent modelling. This suggests that the Bayesian approach to opponent modelling proposed by Ponsen et al. [16] can be a valuable contribution to a poker program. However, there are some limitations to our research including the relatively low number of MCTS iterations (1000) and the relatively low number of games played (10,000) during each of the experiments. Future research is necessary to gain more evidence for our conclusions.