{'fraction_non_alphanumeric': 0.02453271028037383, 'fraction_numerical': 0.0, 'mean_word_length': 5.395522388059701, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Searching Digital Heritage: Putting IR Evaluation in Context

Anyone offering cultural heritage content in a digital library is naturally interested in assessing its performance: how well does my system meet my searchers' information needs? Standard evaluation benchmarks in the Cranfield/TREC paradigm allow us to research the generic retrieval effectiveness of system, by abstracting away from the specific document genre, use-case and searcher stereotype. While this is of clear value, it also ignores the unique content and user community of a particular digital library. How does this content differ, and how do the tasks and searchers differ? Can we make the evaluation tailored to their unique characteristics? Moreover, can we distinguish different types of usage (e.g., the professionals versus the public) and how does this impact the evaluation?