{'fraction_non_alphanumeric': 0.019346230820547032, 'fraction_numerical': 0.0, 'mean_word_length': 5.097560975609756, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Play selection in football : a case study in neuro-dynamic programming

Using a version of the game of American football, this paper presents an extensive computational study of approximate dynamic programming methods. The problem of optimal play selection for the offensive team during a single offensive drive is cast as a stochastic shortest path problem and serves as a medium-scale testbed for algorithms inspired by policy iteration. Our methods are such that at each iteration a new policy is computed on the basis of an approximate evaluation of the policy's reward-to-go function. For a given policy, the approximate evaluation of reward-to-go is obtained either as the solution or as a step toward the solution of a least squares training problem defined by simulated state/reward data pairs. Numerical results for a variety of methods are presented. Because football is a medium-scale problem, a numerical solution can easily be computed, providing a yardstick for meaningful comparisons of the approximate methods. The main purpose of this paper is to illustrate the application of neuro-dynamic programming methods in solving a concrete problem. Along the way, however, we are able to contrast and compare the methodologies both in terms of performance and complexity of implementation. Finally, our numerical results nicely illustrate some the interesting properties of the algorithms that are becoming recognized as "typical behavior" by various researchers in the learning community.