{'fraction_non_alphanumeric': 0.011019283746556474, 'fraction_numerical': 0.005509641873278237, 'mean_word_length': 5.37719298245614, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Intra-cluster training strategy for deep learning with applications to language identification

In this study we address the problem of training a neural network for language identification using speech samples in the form of i-vectors. Our approach involves training a classifier and analyzing the obtained confusion matrix. We cluster the languages by simultaneously clustering the columns and the rows of the confusion matrix. The language clusters are then used to define a modified cost function for training a neural-network that focuses on distinguishing between the true language and languages within the same cluster. The results show enhanced language identification on the NIST 2015 language identification dataset.