{'fraction_non_alphanumeric': 0.013380909901873328, 'fraction_numerical': 0.0, 'mean_word_length': 5.678571428571429, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Studying human judgments of relevance: interactions in context

This paper discusses the ways participants in a two-year ethnographic study judged relevance when engaged in searching and research tasks. Two experienced academics have been observed evaluating informative artefacts (documents, citations or other representations) encountered in the course of their own research projects. This study sought to explore the criteria and clues used to make decisions about the relevance of retrievable items. In presenting some of the findings from this longitudinal study, the paper demonstrates the value of this approach for enhancing our understanding of the evolving nature of human relevance judgments. The paper will describe how this interaction involves not only the notion of searcher-system communication, but a range of encounters that inform and influence that particular communication at the search interface. The paper suggests future collaboration between system specialists and human behaviour specialists to further our understanding of the socio-material systems in which people make judgments of relevance.