{'fraction_non_alphanumeric': 0.01680672268907563, 'fraction_numerical': 0.0021008403361344537, 'mean_word_length': 5.618055555555555, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Detection of paraphrases for Devanagari languages using support vector machine

Paraphrase is a process of computing the semantic similarity between sentences, which are not lexicographically similar. It relates to the writing a sentence in another form. Though a number of metrics for English language have been proposed in literature, to quantify textual similarity; but none for Devanagari language. Existing system for Indian language paraphrase detection uses lexical similarity are supervised and requires large scale tagged corpus. The proposed method employs SVM learning metrics, based on lexicography similarity with producing output as +1 for paraphrased, −1 for not paraphrased, takes a sentence as input and produces another sentence without changing its semantic. In particular, the system addresses the problem for detection of monolingual text to text similarity for fusion language like Hindi and Marathi, which has complex morphology.