{'fraction_non_alphanumeric': 0.02117863720073665, 'fraction_numerical': 0.0, 'mean_word_length': 5.431952662721893, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Efficient Reinforcement Learning in Parameterized Models : The Parameter Elimination Approach

We consider reinforcement learning in a parameterized setup, where the model is known to belong to a compact set of Markov Decision Processes (MDPs), under the discounted return criteria. We propose an on-line algorithm for efficient learning in such parameterized models, the Approximate Parameter Elimination (APEL) algorithm. The algorithm accepts a dense finite grid in the parameter set and employs a modified version of Wald’s sequential probability ratio test to eliminate unlikely grid points, while using an optimistic policy for effective exploration. Our analysis provides PAClike polynomial upper bounds on the total number of suboptimal actions taken. In particular, our worst-case bounds are linear (up to a logarithmic term) in the number of grid points, independently of the cardinality of the state and action spaces. We further demonstrate that much better dependence on the number of grid points is possible, depending on the specific information structure of the problem.