{'fraction_non_alphanumeric': 0.01403061224489796, 'fraction_numerical': 0.0, 'mean_word_length': 4.789667896678967, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Statistical language modeling for personalizing information retrieval

This paper reviews the application of statistical language modeling in information retrieval to make web search process as efficient as possible. IR systems have been widely used for the task of ranking documents accurately for a given keyword query from the vast collection. Typically, web search engines are required to satisfy diverse information needs of user i.e. different users may have different search intention when they submit the same query to a search engine. Since there is a huge amount of user data available, it becomes possible to build user model so as to identify the more relevant documents based on individual users interest. This paper focuses on such a model suitable for IR especially for web search. It requires very less effort from the user side. The proposed user modeling would be helpful when the keyword query issued by the user is short and ambiguous i.e. most of the users provide inaccurate keyword query which is imprecise and they often under-specify their exact information needs. Thus query becomes ambiguous which needs to be understood by the retrieval system. Hence, personalization strategy needs to be adopted in order to solve these problems faced by the retrieval system. In order to personalize web search, the user profile model is formulated by using users browsing history. In the same way, past queries and clicks made by the user can also be used for building user search model. This approach is implemented and tested using real time user data.