{'fraction_non_alphanumeric': 0.04048070841239722, 'fraction_numerical': 0.007590132827324478, 'mean_word_length': 5.1796875, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Non-uniform Region Based Features for Automatic Language Identification

An audio utterance can be identified as being spoken in a particular language by using automatic language identification (LID). Each language has its own phoneme set. Hence combination of these phonemes governed by phonotactics will help in distinguishing languages. In this work, we propose an automatic language identification system utilizing features derived from non-uniform speech regions to represent phonotac-tic differences among 4 Indian languages, namely Malayalam, Marathi, Assamese, and Kannada. For this, broad phoneme labels, namely approximant (A), closure (C), fricatives (F), nasals (N), plosive/stop (P), voiced stop (B), vowels (V), and silence (S) are obtained automatically by a broad phoneme classifier (BPC). It is a DNN-based classifier which uses hand-crafted features and Mel-frequency cepstral coefficients (MFCC). In order to automatically segment speech to smaller regions, first it is chopped at every silence regions using the labels obtained from BPC. Later, it is split again at the end of each vowel. Hence, small non-uniform regions are obtained which contain phoneme combinations that may be specific to the language of the utterance. From each region, only a fixed number of frames containing certain combination of phonemes are selected. A DNN classifier is trained using 13-dimensional MFCC features of 12 fixed frames of non-uniform regions for performing LID. An average accuracy of 97.03% is obtained for test utterances of 10 sec duration belonging to 4 languages.