{'fraction_non_alphanumeric': 0.04024144869215292, 'fraction_numerical': 0.008048289738430584, 'mean_word_length': 4.354838709677419, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Proof of Lemma 2

For simplicity, our derivation of relabeling in Section 4 assumed that entire trajectories were provided. This section outlines how to do relabeling with inverse RL when we are only provided with (s, a, s′) transitions, rather than entire trajectories. This derivation will motivate the use of the soft Q-function in Eq. 8. In this case, policy distribution q in the MaxEnt RL objective (Eq. 6) is conditioned on the current state and action (st, at) in addition to the task ψ: