{'fraction_non_alphanumeric': 0.020558002936857563, 'fraction_numerical': 0.0, 'mean_word_length': 5.123595505617978, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Shape Game A Multimodal Game Featuring Incremental Understanding

The goal of much research in speech understanding systems is to design interfaces which allow for humans to interact with computers in a more natural and intuitive fashion. Intuitively, speech is a very natural, convenient, and fast way of communicating information – and people produce and understand speech with little overt effort. Despite the ease with which humans communicate with one another using speech, many find it frustrating to communicate with computers via speech. This frustration can be linked to many limitation of speech understanding systems, including limitations in the accuracy of automatic speech recognition, parsing, referent resolution, and a general lack of world knowledge. But even if these challenges were overcome, the vast majority of speech understanding systems would likely still feel quite unnatural, owing to the strong turn-based model that most are designed around. Typically, speech understanding systems are designed following a model in which users are expected to formulate complete, well-formed ideas (typically in the form of commands or queries) in a fully fluent utterance. The reality is, most people don’t talk this way – those of us who can clearly stand out: television personalities like American Idol host Ryan Seacrest or news anchors like Tom Brokaw come to mind. The reality is, most of us aren’t very good at staring at a television camera – or a computer monitor – and speaking fluently, without immediate feedback that anyone (or anything) is listening or understanding. In my experience designing and testing several different multimodal speech understanding systems, this strict idea of turn taking has always stood out as being extremely awkward whenever naive users try to interact with a system. One of the most challenging aspects of designing speech understanding systems is giving a sense of what the system can understand, especially conveying to a user why the system has made an error in understanding. Humans are quite good at conveying what they have or have not understood about what another person has said, and one strong mechanism for signaling this is through immediate feedback to a person as they are speaking. This feedback comes in many different forms, including gestures, facial expressions, and backchanelling; and it can play a vital role in helping a speaker know when to rephrase, speak more slowly, and so on. However, the strict turn-taking structure of most speech understanding systems makes this sort of subtle, yet critical, interaction impossible. After attempting to formulate a fluent utterance, users are left to then try to figure out what went wrong and why.