{'fraction_non_alphanumeric': 0.018917896329928113, 'fraction_numerical': 0.0022701475595913734, 'mean_word_length': 5.4330900243309, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Improving semantic similarity measures for word pair comparison

The semantic web provides a common framework that allows data to be shared and reused across application, enterprise, and community boundaries. In order to achieve the goals of the semantic web, it have to be able to define and to describe the relations among data (i.e., resources) on the Web. Ontologies are one of the formal representation for organizing information in the semantic web and they are also used in artificial intelligence, systems engineering, software engineering, biomedical informatics, library science, enterprise bookmarking, and information architecture as a form of knowledge representation about the world or some part of it. In the semantic web context, since many actors provide their own ontologies, ontology matching or ontology alignment has taken a critical role for helping heterogeneous resources to inter-operate [23]. Ontology matching tools find classes of data that are “semantically equivalent”. This process determine correspondences between concepts which are called alignments [22]. Finding those correspondences imply a semantic similarity assessment between the involved concepts. Semantic similarity of words pairs is often represented by the similarity between the concepts associated with the words. Several methods have been developed to compute words similarity, most of them operating on taxonomic dictionaries like WordNet [24] or external corpus like the Brown Corpus. However the majority of them suffer from a serious limitation. They only focus on the semantic information shared by those words, or in the semantic differences, but they have been rarely combined in a broader perspective. In this thesis we developed and applied a model of semantic similarity computation for word pair comparison. This model consider the semantic commonalities and the semantic differences as the core of its approach. By applying the model five new WordNet-based semantic similarity measures for word pair comparison were created. Four of this semantic similarity measures obtained higher values of correlation with human judgment than their original expressions, while the fifth one remained as competitive as their original version. We also study WordNet taxonomic properties to extend a corpus-independent information content metric. The application of this new metric in one of the previously developed node-based semantic similarity allowed us to obtain the highest value of correlation with respect to human judgment. This thesis provides a general an extensible approach of semantic similarity computation for word pair comparison.