{'fraction_non_alphanumeric': 0.011931394481730051, 'fraction_numerical': 0.0, 'mean_word_length': 5.212962962962963, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Efficiency Analysis Of Resource Request Patterns In Classification Of Web Robots And Humans

The paper deals with the problem of classification of Web traffic generated by robots and humans on e-commerce websites. Due to the still growing proliferation and specialization of bots, a large body of research into characterization and recognition of their traffic has been conducted so far. In particular, some approaches to classify bot and human sessions on websites have been proposed in the literature. In this paper we verify and discuss the efficiency of such recently proposed approach, which uses differences in resource request patterns of bots and humans. We reconstructed Web sessions from actual HTTP log data for three different ecommerce sites, varying in the traffic intensity and proportions of bot sessions in the overall traffic. Two heuristic procedures for labeling sessions as driven by a bot or a human were proposed and implemented. Resource request patterns for both session classes, using both session labeling procedures, were analyzed and their potential to differentiate between bot and human sessions was investigated. Results show that the broader session labeling procedure allows one to capture more bot sessions and that resource requests patterns are a good discriminant of bots and humans on e-commerce sites.