{'fraction_non_alphanumeric': 0.017205422314911366, 'fraction_numerical': 0.0010427528675703858, 'mean_word_length': 5.279869067103109, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Adapting information retrieval to user needs in an evolving web environment

The booming growth of digitally available information has thoroughly increased the popularity of search engine technology over the past years. At the same time, upon interacting with this overwhelming quantity of data, people usually expect search results relevant to their current task. It is thus very important to utilize high quality personalization methods which efficiently target the short user query toward the real information need. With the increasing popularity of Web 2.0 sites, the amount of content available online is again multiplying at a rapid rate, at the same time becoming also more diverse in terms of content types – pictures, music, Web pages, etc. – and quality. On the other hand, collaborative tagging has become an increasingly popular means for sharing and organizing Web resources, leading to a huge amount of user generated metadata. Yet analyses show that there are huge differences between the tagging and the querying vocabularies, such that queried terms are underrepresented in the annotations of the resources. Thus, semantically enriching resources’ annotations becomes crucial for efficient retrieval. In this thesis we propose solutions for several issues which arose over time as the Web and Information Retrieval evolved. By performing experiments with a log of 2.4 million queries we create a model for Web query reformulation processes. We use the variation in Query Clarity, as well as the Part-Of-Speech pattern transitions as indicators of users’ search actions and are thus able to provide interesting insights into users’ Web behavioral patterns. We choose to follow Query Expansion patterns and propose to personalize Web queries by expanding them with terms collected from each user’s Personal Information Repository. We introduce five broad techniques for generating the additional query keywords by analyzing user data at increasing granularity levels, ranging from term and compound level analysis up to global co-occurrence statistics, as well as to using external thesauri. We then extend the application of our algorithms to Just-In-Time IR systems. Software agents collect and analyze the users’ active personal desktop documents and recommend URLs relevant to the users’ current work. Our extensive empirical analysis under five different scenarios shows these approaches to perform very well, producing a strong increase in the quality of the output rankings. We study the usefulness of collaborative tagging for identifying which characteristics of the objects are predominantly described and what kinds of tags are employed across multiple domains and resource types. By performing a similar analysis on user queries we identify the gaps between the tag space and the querying vocabulary. We then try to bridge the identified gaps, focusing in particular on multimedia resources. We concentrate on the two scenarios of music and picture resources and develop algorithms which identify usage (theme) and opinion (mood) characteristics of the items. The mood and theme labels our algorithms infer are recommended to the users, in order to support them during the annotation process. Moreover, our algorithms are also able to exploit the social information produced by users in form of tags, titles and photo descriptions, for classifying pictures into different event categories. This allows browsing and organization of picture collections in a natural way, by events. The extensive evaluation of the proposed methods against user judgments, as well as against expert ground truth reveal the high quality of our recommended annotations. We also provide insights into possible extensions for music and picture tagging systems to support retrieval and open new possibilities for multimedia retrieval.