{'fraction_non_alphanumeric': 0.020689655172413793, 'fraction_numerical': 0.008045977011494253, 'mean_word_length': 5.548872180451128, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Using local & global phonotactic features in Chinese dialect identification

Conventional techniques for spoken language identification use variants of phone similarity and language model scoring, which represent local phonetic constraints in spoken languages. We explore the identification of Chinese dialects which share the same written script and have similar sound systems and syllable structures. As such, local phonetic constraints do not provide enough discriminative information among dialects. We propose to use latent semantic analysis (LSA) to extract global features that represent the high-order statistics in the cooccurrence of sounds. Experiments show that we can achieve the best performance by combining acoustic, n-gram language modeling and LSA scores. An accuracy of 99.23% is achieved in 4-way classification tests using 20-second speech sessions.