{'fraction_non_alphanumeric': 0.01859504132231405, 'fraction_numerical': 0.0, 'mean_word_length': 6.021739130434782, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Counterfactual Data-Fusion for Online Reinforcement Learners

Online learning agents possess the capacity to learn from both personalized experimentation as well as the observed behaviors of other agents interacting with the environment. However, data collected through these different modalities may not be naively combined due to changes in the decisionmaking context, including factors that may be unobserved. The data-fusion problem addresses how information collected under such disparate conditions (observationally, experimentally, and counterfactually) can be combined to yield more informative results than the independent datasets alone. The present work provides a recipe for combining multiple datasets to accelerate learning in a variant of the Multi-Armed Bandit problem with Unobserved Confounders (MABUC). We demonstrate this data-fusion approach through an enhanced Thompson Sampling bandit player, and support its efficacy with extensive simulations.