{'fraction_non_alphanumeric': 0.015151515151515152, 'fraction_numerical': 0.0030303030303030303, 'mean_word_length': 5.63819095477387, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A performance evaluation of similarity measures, document term weighting schemes and representations in a Boolean environment

This chapter reports on the results of a study of the effectiveness of ranking algorithms. Also reported here will be some unexpected findings relating to the performance of document representations and searcher differences. These findings are a by-product of the evaluation of the ranking algorithm. The goal of the study was to evaluate ranking algorithms so that generalisations about their effectiveness could be made. Many different ranking algorithms have been suggested (Sager and Lockemann, 1976). Evaluation of the effectiveness of these algorithms has been conducted under differing experimental conditions. These differences in the evaluation conditions have made comparisons of ranking algorithms uncertain. This study evaluated the effectiveness of the ranking algorithms using a single database, common user population, and common sets of queries and relevance judgements. This approach allowed the relative effectiveness of the ranking algorithms to be determined. It is felt, but does need further examination, that while the absolute effectiveness of any ranking algorithms may vary with the environment, the relative effectiveness of the ranking algorithms will be invariant.