{'fraction_non_alphanumeric': 0.026771653543307086, 'fraction_numerical': 0.0, 'mean_word_length': 5.115384615384615, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Duplication removal for a search-based recommendation system

A search-based recommendation system looks, in the code repository, for programs that are relevant to the program being edited. Storing a large amount of open source programs into the repository will make the search results better, but also causes the code clone problem; i.e., recommending a set of program fragments that are almost idential. To tackle this problem, we propose a novel approach that ranks recommended programs by taking their “freshness” count into account. This short paper discusses the background of the problem, and illustrates the proposed algorithm.