{'fraction_non_alphanumeric': 0.01832460732984293, 'fraction_numerical': 0.0, 'mean_word_length': 5.78698224852071, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Learning Phonological Categories by Independent Component Analysis*

Abstract This work aims at discovering, in an unsupervised fashion, the nature of phonemes on the basis of their distributional information within a representative corpus. We focus on some basic issues of the phonology tradition such as the vowel/consonant distinction, the configuration of natural phonological classes and the identification of phonologically-motivated constraints such as the vowel harmony. The simulations use a matrix decomposition method, the so-called independent component analysis (ICA), which is able to find latent factors underlying a set of multivariate observations. We designed three different corpora (English, Italian and Finnish) as input to the system in order to test the robustness of the methodology and the consistency of the results. The results are also investigated by means of self-organizing map mappings. The work emphasizes the exploitation of distributional information and its effectiveness in discovering the inherent phonotactic regularities of a given language and generalizing phonological behaviours from these regularities.