{'fraction_non_alphanumeric': 0.016037735849056604, 'fraction_numerical': 0.0, 'mean_word_length': 5.430303030303031, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
TIDE : A Testbed for Interactive Spoken Dialogue System Evaluation

Telephone-based spoken dialogue platforms usually log a multitude of information for each interaction between user and system. Due to its large amount and specificity, this information is very difficult to interpret. In this paper, we present a new testbed for the analysis and evaluation of log-file information collected with spoken dialogue platforms. On the signal level, parameters are extracted which allow the sources of recognition errors to be allocated to channel and user characteristics. On the symbolic level, phonemic similarities are computed which allow the confusability of lexicon and grammar to be evaluated. The algorithms are integrated into a graphical user interface for an effective analysis of system performance. It is evaluated on data collected within a usability test of a prototype implementation for an automated pre-qualifying application of Deutsche Telekom, and correlations between extracted parameters, recognition rates and user judgments are pointed out.