{'fraction_non_alphanumeric': 0.018731988472622477, 'fraction_numerical': 0.002881844380403458, 'mean_word_length': 5.95, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Deduplicação de Contatos em Dispositivos Móveis Utilizando Similaridade Textual e Aprendizado de Máquina [Contacts Deduplication in Mobile Devices Using Textual Similarity and Machine Learning]

This paper presents a method that identifies duplicate contacts, i.e., records representing the same person or organization, automatically collected from multiple data sources. Contacts are compared using several similarity functions, of which scores are combined by a classification model based on decision trees, which eliminates the need for an expert to manually configure similarity thresholds. The experiments show that the proposed method correctly identified up to 92% of duplicate contacts.