{'fraction_non_alphanumeric': 0.0196078431372549, 'fraction_numerical': 0.004668534080298786, 'mean_word_length': 5.961038961038961, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Advances in Feature Extraction and Modelling for Short Duration Language Identification

This paper presents an overview of the progression of short duration spoken language identification systems and current developments. It reviews different language identification architectures including single level, hierarchical, fully connected, convolutional neural network and bidirectional long-short term memory. The work presented in this paper aims to explore the most effective language identification architecture, frontend and backend for the short duration classification task. Specifically, the use of frequency domain linear prediction features is proposed with bidirectional long short-term memory recurrent neural networks for language identification, which aims to model temporal dependencies between past and future frame based features in short utterances. It shows significant improvements of 26.1% in terms of Cavg compared to state-of-the-art i-vector approach. Evaluations of the LID systems are presented using AP17-OLR language recognition evaluation tasks.