{'fraction_non_alphanumeric': 0.025133910177173466, 'fraction_numerical': 0.0057684384013185, 'mean_word_length': 4.62037037037037, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Efficiency improvements for interactions of web-agents

Interactions of web-based agents vary in time and are greatly influenced by hardware on both sides, the networks used, and the operating system’s software used. As interaction completion times vary in time, they can be described as random variables. Knowledge about this random process is very useful to improve performance. Knowing the distribution an agent can come up with a maximum time it is willing to wait for a reply to its request. This choice has to be taken carefully, as the request will have to be reissued to the same or another agent hopefully with a better response time. This performance improvement is known as restart method [4, 3, 1]. It is often encountered in practice when browsing the web: a web page takes ages to be retrieved. After some time users impatiently reload the page and it is promptly downloaded. The principle underlying this example is general in nature. Given a client and a server, the client issues a request to the server. If the request is not served after a certain amount of time, the client reissues the request, possibly to another server. Using this strategy, the client expects an overall improvement, if the timeout period is set correctly. How can this be achieved? First of all data needs to be collected to find out the distribution of completed requests. For certain distributions [1] a cutoff can be chosen, which reduces the mean and variance of the new distribution in comparison to the one, which does not employ the restart method. An example of a distribution, where the restart method works, is the lognormal distribution [3, 4]. It is asymmetrical and after a peak the distribution slowly tails off. Therefore performance improvements can be expected if requests are timed out after the peak. Huberman et al. [4, 3] investigated the restart method for UDPconnections and retrieval of index web pages, where they found lognormal distributions in the data. We verified this experiment as shown in Fig. 1 A,B, where UDP packets sent from the UK to Japan clearly show a lognormal distribution with mean 9.25 and variance of 0.5. While Huberman et al. are mainly interested in social processes influencing network traffic, our work aims to put the restart method to practice. This requires in particular a detailed analysis of the different phases of agent interactions, which is not considered by Huberman et al.