{'fraction_non_alphanumeric': 0.015989340439706862, 'fraction_numerical': 0.0, 'mean_word_length': 5.080971659919029, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Learning User Models with Limited Reinforcement: An Adaptive Human-Robot Interaction System

Spoken dialog allows for very natural human-robot interaction, but natural dialog can lead to considerable uncertainty during the dialog as a result of ambiguous phrases or noisy speech recognition. Planning algorithms such as the Partially Observable Markov Decision Process (POMDP) have been used successfully to overcome this uncertainty and generate reasonable interactions during natural dialogs between people and mobile robots. However, like all dialog systems, a POMDP is defined by a large number of parameters that may be difficult to specify a priori from domain knowledge. Even with an online adaptive system, learning the parameters may require a tedious training period from the user. In this paper, we present an approach which lets the agent decide when it needs more information to be an effective dialog manager. If the agent feels that it is familiar with a situation, it acts based on its current understanding of the dialog. However, when faced with an unfamiliar scenario, it will ask its human user what he or she would do in the agent’s situation— advice that we believe is relatively easy for humans to give. Our approach avoids a training period of constant questioning by allowing the agent to discover the consequences of a poor decision without actually making mistakes. We demonstrate our approach both in simulation and on a dialog manager for a robotic wheelchair application.