{'fraction_non_alphanumeric': 0.021052631578947368, 'fraction_numerical': 0.004678362573099415, 'mean_word_length': 5.388059701492537, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Efficient Exploration for Constrained MDPs

Given a Markov Decision Process (MDP) defined by a simulator, a designated starting state s0, and a downside risk constraint defined as the probability of reaching catastrophic states, our goal is to find a stationary deterministic policy π that with probability 1 − δ achieves a value V (s0) that is within of the value of the optimal stationary deterministic νfeasible policy, V ∗(s0), while economizing on the number of calls to the simulator. This paper presents the first PAC-SafeRL algorithm for this purpose. The algorithm extends PACRL algorithms for efficient exploration while providing guarantees that the downside constraint is satisfied. Experiments comparing our CONSTRAINEDDDV algorithm to baselines show substantial reductions in the number of simulator calls required to find a feasible policy.