{'fraction_non_alphanumeric': 0.01957446808510638, 'fraction_numerical': 0.010212765957446808, 'mean_word_length': 5.288770053475936, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
AMRITA_CEN@FIRE-2014: Morpheme Extraction and Lemmatization for Tamil using Machine Learning

This paper presents the method of Morpheme Extraction and lemmatization for Tamil language in Morpheme Extraction Task (MET) of FIRE-2014. Tamil is a morphologically rich and agglutinative language. Such a language needs deeper analysis at the word level to capture the meaning of the word from its morphemes and its categories. In this attempt, the methodology employed to extract Tamil morphemes and lemmas are based on a supervised machine learning algorithm for nouns and verbs and simple suffix stripping for pronouns and proper nouns. Morphemes are extracted for other Part-of-Speech categories using Tamil Part of Speech tagger. In supervised learning, Morphological analyzer problem is redefined as a classification problem. We decompose the problem of noun and verb morpheme extraction into two sub-problems: learning to perform morpheme identification of words in a text, and learning to perform morpheme tagging. In addition to the Morpheme extraction task results of FIRE-2014, we have carried out different experiments to show the effectiveness of the proposed method.