{'fraction_non_alphanumeric': 0.011157601115760111, 'fraction_numerical': 0.005578800557880056, 'mean_word_length': 5.648148148148148, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Unsupervised semantic classification methods

A current problem in text processing is the inability to make accurate unsupervised semantic classification systems. In this research we study the unsupervised semantic classification problem using several approaches. We find that morphological and semantic hints can be translated into effective rules within semantic classification. Our results showed a 66% recall rate and a 70% precision rate. We also observed that using raw contextual words as a metric for observing similarity between concepts is minimally effective. Finally we propose further research topics that may be able to improve recall and precision rates of unsupervised semantic classification systems.