{'fraction_non_alphanumeric': 0.011345218800648298, 'fraction_numerical': 0.0016207455429497568, 'mean_word_length': 5.206030150753769, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An Analysis of User Strategies for Examining and Processing Ranked Lists of Documents

The predominant display of document retrieval results is a ranked list of query-biased summaries. When examining and processing search results, users must make complex decisions about how to allocate their time and relevance judging effort between evaluation of summaries and the full documents reachable with a mouse click on a summary. We performed a cluster analysis of the search results processing behavior of 48 user study participants. This analysis produced three clusters of behavior. Across these clusters, we found participants to employ two approaches to the evaluation of summaries: fast and liberal, or slow and neutral. Liberal judges of summaries tolerate clicking on non-relevant summaries in an effort to avoid missing relevant documents. The best performing group employed a fast and liberal approach to summaries and also quickly evaluated full documents. The two lower performing groups utilized opposite strategies with one group trading a short amount of time on the summaries for more time spent evaluating full documents and the other group invested more time in evaluating summaries and spent a shorter time on documents.