{'fraction_non_alphanumeric': 0.033027164354113736, 'fraction_numerical': 0.0070353722884502635, 'mean_word_length': 5.519745222929936, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 4, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
How Authority-Related Epistemological Beliefs and Salience of Source Information Influence the Evaluation of Web Search Results – An Eye Tracking Study

How Authority-Related Epistemological Beliefs and Salience of Source Information Influence the Evaluation of Web Search Results – An Eye Tracking Study Yvonne Kammerer (y.kammerer@iwm-kmrc.de) Knowledge Media Research Center, Konrad-Adenauer-Strasse 40, 72072 Tuebingen, Germany Evelin Wollny (e.wollny@iwm-kmrc.de) Knowledge Media Research Center, Konrad-Adenauer-Strasse 40, 72072 Tuebingen, Germany Peter Gerjets (p.gerjets@iwm-kmrc.de) Knowledge Media Research Center, Konrad-Adenauer-Strasse 40, 72072 Tuebingen, Germany Katharina Scheiter (k.scheiter@iwm-kmrc.de) Knowledge Media Research Center, Konrad-Adenauer-Strasse 40, 72072 Tuebingen, Germany Abstract Evaluating topicality and information quality during web search In this study we examined how authority-related epistemological beliefs (AREBs) affect the evaluation of web search results presented by a search engine. Moreover, we investigated whether increasing the salience of source information on a search engine result page fosters students’ evaluation processes, and whether the salience of source information moderates the effects of AREBs. Thirty university students participated in a web search experiment addressing a controversial search topic. Participants either used a standard Google search result list or an augmented search result list additionally containing source categories for each search result. Eye movements and mouse clicks were captured during information search. The results showed that the effects of participants’ AREBs on their gaze behavior were moderated by the salience of source information. Furthermore, salience of source information affected participants’ selection behavior, such that with additionally presented source categories they were more likely to select search results linked to web portals. Keywords: web search; epistemological beliefs; search engines; evaluation processes; interface design; eye tracking; HCI Introduction In recent years, the World Wide Web (WWW) has evolved into one of the most important public sources for science- related contents offering convenient and rapid access to vast amounts of information. Particularly for domains of personal concern such as medicine and healthcare using the web has achieved great popularity. At the same time, however, the web imposes new challenges onto searchers due to its high complexity and heterogeneity of information. Search engines help to find information on the web, but searchers are still required to evaluate and select search results for further inspection of corresponding websites. Thus, search result descriptions displayed on search engine result pages (SERPs) play a critical role in guiding search processes as searchers rely on their evaluations of these descriptions to decide whether to visit the web pages they represent or not. In cognitive science, a central approach used to explain and predict search result selection as well as other types of web search behavior is the Information Foraging Theory (Pirolli & Card, 1999). The theory assumes that the selection of hyperlinks (e.g., from a SERP), is determined by their so called “information scent”. Information scent is defined as the semantic similarity between a cognitive representation of the current information needs and an external representation of cues on a display (e.g., keywords in the title, abstract, or URL of a search result). The stronger the semantic overlap between the cues contained in a particular search result and the user’s information needs (i.e., the stronger the topical fit), the stronger is the information scent of the search result. The information scent of a particular search result determines its likelihood for being selected. Thus, according to Information Foraging Theory, search result selection is mostly guided by evaluating the topicality of web information. In line with this theory, thinking-aloud data from web search experiments on science-related topics revealed that university students strongly focus on the topical fit (i.e., connection to the task) when evaluating search results returned by a search engine (e.g., Google). Evaluation criteria beyond topicality with regard to the quality of information (e.g, credibility, trustworthiness, reliability, or accuracy) were uttered rather seldom (Brand-Gruwel, van Meeuven, & van Gog, 2008; Gerjets, Kammerer, & Werner, subm.). Although these findings are in line with the Information Foraging Theory, they are nevertheless rather astonishing if one takes into account that anyone can publish any information on the web. As a result, the web is characterized by a high heterogeneity of information sources, varying not only with regard to topical fit, but also with regard to authority and credibility. For instance, when it comes to information on complex and contradictory scientific topics, not only scientific and other institutions, but also journalists, companies, and laypeople can act as information providers.