{'fraction_non_alphanumeric': 0.011637572734829594, 'fraction_numerical': 0.0, 'mean_word_length': 5.615384615384615, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Reinforcement Learning with Deep Architectures

There is both theoretical and empirical evidence that deep architectures may be more appropriate than shallow architectures for learning functions which exhibit hierarchical structure, and which can represent high level abstractions. An important development in machine learning research in the past few years has been a collection of algorithms that can train various deep architectures effectively. These methods have already led to many successes in the areas of supervised and unsupervised learning. They may prove to be just as useful in reinforcement learning as well, since solving a reinforcement learning problem depends on effectively approximating one or more state-value functions, which in general are just as likely to exhibit hierarchical structure as functions encountered in other settings. In this paper, we consider some of the issues that arise when trying to integrate ideas from deep learning into the reinforcement learning framework, present a class of algorithms which we refer to as Iterative Feature Extracting Learning Agents (IFELAs), and compare their performance on the inverted pendulum problem to more standard approaches.