title: How to properly make requests per second (RPS) Test with aiohttp client and asyncio? Unexpected RPS number with small asyncio.sleep argument
body: <p>I need to make requests per second (RPS) test of some http server. I wrote the code below to do the task. The code was inspired by leaky bucket algorythm. Code works well, but I found that when I try to send requests every (less than) 1/60 seconds, I start getting unexpected number of requests per minute (more than 100 000). I assume that problem might be related to <code>asyncio.sleep</code> precision, when argument is too small it is considered as equal to 0 probably. Test OS: Windows 10 Home. Tested in Pycharm IDE.</p>
  <p><strong>Test Results</strong>:</p>
  <pre><code>INFO | 2021-03-24 18:40:04,056 | Exp. Rate 10 req/sec Sent: 559 Received: 559 Error: 0 Ok: 559 Exceeded: 64 Cancelled: 0
  INFO | 2021-03-24 18:41:04,046 | Exp. Rate 20 req/sec Sent: 980 Received: 980 Error: 0 Ok: 980 Exceeded: 82 Cancelled: 0
  INFO | 2021-03-24 18:42:04,048 | Exp. Rate 30 req/sec Sent: 1329 Received: 1329 Error: 0 Ok: 1329 Exceeded: 142 Cancelled: 0
  INFO | 2021-03-24 18:43:04,058 | Exp. Rate 40 req/sec Sent: 2074 Received: 2074 Error: 0 Ok: 2074 Exceeded: 147 Cancelled: 0
  INFO | 2021-03-24 18:44:04,058 | Exp. Rate 50 req/sec Sent: 2074 Received: 2074 Error: 0 Ok: 2074 Exceeded: 163 Cancelled: 0
  INFO | 2021-03-24 18:45:04,033 | Exp. Rate 60 req/sec Sent: 2139 Received: 2138 Error: 0 Ok: 2138 Exceeded: 213 Cancelled: 0
  # what happened here?
  INFO | 2021-03-24 18:46:04,038 | Exp. Rate 70 req/sec Sent: 115784 Received: 110788 Error: 0 Ok: 110788 Exceeded: 110421 Cancelled: 0
  INFO | 2021-03-24 18:47:04,033 | Exp. Rate 80 req/sec Sent: 110109 Received: 110151 Error: 0 Ok: 110151 Exceeded: 110151 Cancelled: 0
  INFO | 2021-03-24 18:48:04,033 | Exp. Rate 90 req/sec Sent: 109617 Received: 109578 Error: 0 Ok: 109578 Exceeded: 109578 Cancelled: 0
  INFO | 2021-03-24 18:49:04,103 | Exp. Rate 100 req/sec Sent: 109056 Received: 109359 Error: 0 Ok: 109359 Exceeded: 109359 Cancelled: 0
  </code></pre>
  <p><strong>Test Server code</strong> nothing special here:</p>
  <pre class="lang-py prettyprint-override"><code>from aiohttp import web
  from random import randint, seed
  import asyncio
  import logging
  
  
  async def hello_world(req: web.Request) -&gt; web.Response:
      # await asyncio.sleep(randint(1, 2))  # simulate network + server delay
      # imagine there is nearly no server delay, just network delay
      return web.json_response({&quot;hello&quot;: &quot;world&quot;})
  
  if __name__ == '__main__':
      _format = &quot;%(levelname)s | %(asctime)s | %(message)s&quot;
      logging.basicConfig(level=logging.INFO, format=_format)
      seed(1)
      app = web.Application()
      app.router.add_get(&quot;/&quot;, hello_world)
      web.run_app(app, host=&quot;0.0.0.0&quot;, port=7779)
  </code></pre>
  <p><strong>Client - RPS Tester</strong> code:</p>
  <pre class="lang-py prettyprint-override"><code>import asyncio
  import logging
  from aiohttp import ClientSession
  from time import monotonic
  
  URL = &quot;http://localhost:7779&quot;
  START_RATE = 60  # requests per second
  MAX_DELAY = 0.1
  
  
  class RpsCounter:
      &quot;&quot;&quot;Contains all statistics regarding RPS&quot;&quot;&quot;
      def __init__(self):
          self._n_send = 0
          self._n_received = 0
          self._n_error = 0
          self._n_ok = 0
          self._n_delay_exceeded = 0
          self._n_cancelled = 0
  
      def incr_n_send(self):
          self._n_send += 1
  
      def incr_n_cancelled(self):
          self._n_cancelled += 1
  
      def incr_on_received(self, was_error=False, was_exceeded=False):
          self._n_received += 1
          if was_error:
              self._n_error += 1
          else:
              self._n_ok += 1
              if was_exceeded:
                  self._n_delay_exceeded += 1
  
      def _clear(self):
          self._n_send = 0
          self._n_received = 0
          self._n_error = 0
          self._n_ok = 0
          self._n_delay_exceeded = 0
          self._n_cancelled = 0
  
      def display(self, rate: int):
          &quot;&quot;&quot;shows statistics and them clear results for new round&quot;&quot;&quot;
          logging.info(
              (
                  f&quot;Exp. Rate {rate} req/sec&quot;
                  f&quot; Sent: {self._n_send}&quot;
                  f&quot; Received: {self._n_received}&quot;
                  f&quot; Error: {self._n_error}&quot;
                  f&quot; Ok: {self._n_ok}&quot;
                  f&quot; Exceeded: {self._n_delay_exceeded}&quot;
                  f&quot; Cancelled: {self._n_cancelled}&quot;
              )
          )
          self._clear()
  
  
  class AsyncClientOps:
      def __init__(self, start_rate: int, url, counter: RpsCounter, max_delay: float):
          # distribute tasks among workers which send requests
          self._rps_q = asyncio.Queue(maxsize=20)  # defencive behavior
          # controls max number of requests per second (rps)
          self._rate = start_rate  # human readable value
          # used in _throttler asyncio.sleep :
          self._rps_interval = round(1 / self._rate, 4)
          self._url = url
          self._counter = counter
          self._max_delay = max_delay
  
      @property
      def rps_interval(self):
          return self._rps_interval
  
      async def _throttler(self):
          &quot;&quot;&quot;Limit request per second to a certain number. Inspired by 'leaky bucket' &quot;&quot;&quot;
          while True:
              if not self._rps_q.full():  # avoid blocking
                  # can put anything - it is just signal for worker to start
                  await self._rps_q.put(1)
              await asyncio.sleep(self.rps_interval)
  
      async def _send_requests(self, session: ClientSession):
          &quot;&quot;&quot;just sends request to server and change counter number&quot;&quot;&quot;
          while True:
              await self._rps_q.get()
              st = monotonic()
              self._counter.incr_n_send()  # not really sent yet
              try:
                  res = await asyncio.wait_for(self._send_request_inner(session), timeout=5)
              except (asyncio.CancelledError, asyncio.TimeoutError):
                  self._counter.incr_n_cancelled()
              else:
                  if res != 200:
                      self._counter.incr_on_received(was_exceeded=False, was_error=True)
                  else:
                      if monotonic() - st &gt; self._max_delay:
                          self._counter.incr_on_received(was_exceeded=True, was_error=False)
                      else:
                          self._counter.incr_on_received(was_exceeded=False, was_error=False)
  
      async def _send_request_inner(self, session):
          &quot;&quot;&quot;inner task which is cancelled if timed out&quot;&quot;&quot;
          async with session.get(self._url) as response:
              if response.status == 200:
                  await response.json()
                  return 200
              else:
                  return response.status
  
      async def _show_results(self):
          &quot;&quot;&quot;Task which shows statistics every n seconds&quot;&quot;&quot;
          while True:
              await asyncio.sleep(60)
              self._counter.display(self._rate)
              self._rate += 10
              self._rps_interval = round(1/self._rate, 4)
  
      async def async_main(self):
          &quot;&quot;&quot;wrapper around all async ops&quot;&quot;&quot;
          async with ClientSession() as session:
              await asyncio.gather(
                  self._throttler(),
                  self._show_results(),
                  *[self._send_requests(session) for _ in range(5000)]
              )
  
  
  if __name__ == '__main__':
      loop = asyncio.get_event_loop()
      _format = &quot;%(levelname)s | %(asctime)s | %(message)s&quot;
      c = RpsCounter()
      logging.basicConfig(level=logging.INFO, format=_format)
      try:
          async_ops = AsyncClientOps(START_RATE, URL, c, MAX_DELAY)
          loop.run_until_complete(async_ops.async_main())
      finally:
          loop.close()
  
  </code></pre>
  <p><strong>EDIT</strong></p>
  <p>If I make the following small changes, the RPS measure client will be working with much higher RPS. From my point of view, the fact indicates that the problem is related to the event loop speed or to <code>asyncio.sleep</code> precision.</p>
  <pre class="lang-py prettyprint-override"><code>    async def _throttler(self):
          &quot;&quot;&quot;Limit request per second to a certain number. Inspired by 'leaky bucket' &quot;&quot;&quot;
          while True:
              if not self._rps_q.full():  # avoid blocking
                  # can put anything - it is just signal for worker to start
                  # let's send 10 requests one by one
                  for _ in range(10):
                      await self._rps_q.put(1)
              # let's increase sleep interval too
              await asyncio.sleep(self.rps_interval * 10)
  </code></pre>
  <p><strong>New results:</strong></p>
  <pre><code>INFO | 2021-03-24 21:34:15,051 | Exp. Rate 60 req/sec Sent: 3340 Received: 3340 Error: 0 Ok: 3340 Exceeded: 10 Cancelled: 0
  INFO | 2021-03-24 21:35:15,062 | Exp. Rate 70 req/sec Sent: 3980 Received: 3980 Error: 0 Ok: 3980 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:36:15,049 | Exp. Rate 80 req/sec Sent: 4420 Received: 4420 Error: 0 Ok: 4420 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:37:15,065 | Exp. Rate 90 req/sec Sent: 5040 Received: 5040 Error: 0 Ok: 5040 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:38:15,084 | Exp. Rate 100 req/sec Sent: 5760 Received: 5760 Error: 0 Ok: 5760 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:39:15,096 | Exp. Rate 110 req/sec Sent: 5790 Received: 5790 Error: 0 Ok: 5790 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:40:15,106 | Exp. Rate 120 req/sec Sent: 6770 Received: 6770 Error: 0 Ok: 6770 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:41:15,119 | Exp. Rate 130 req/sec Sent: 6810 Received: 6810 Error: 0 Ok: 6810 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:42:15,123 | Exp. Rate 140 req/sec Sent: 8180 Received: 8180 Error: 0 Ok: 8180 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:43:15,130 | Exp. Rate 150 req/sec Sent: 8230 Received: 8230 Error: 0 Ok: 8230 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:44:15,133 | Exp. Rate 160 req/sec Sent: 8200 Received: 8200 Error: 0 Ok: 8200 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:45:15,142 | Exp. Rate 170 req/sec Sent: 8230 Received: 8230 Error: 0 Ok: 8230 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:46:15,144 | Exp. Rate 180 req/sec Sent: 10180 Received: 10180 Error: 0 Ok: 10180 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:47:15,139 | Exp. Rate 190 req/sec Sent: 10590 Received: 10590 Error: 0 Ok: 10590 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:48:15,151 | Exp. Rate 200 req/sec Sent: 10440 Received: 10440 Error: 0 Ok: 10440 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:49:15,165 | Exp. Rate 210 req/sec Sent: 10580 Received: 10580 Error: 0 Ok: 10580 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:50:15,173 | Exp. Rate 220 req/sec Sent: 10530 Received: 10530 Error: 0 Ok: 10530 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:51:15,160 | Exp. Rate 230 req/sec Sent: 10710 Received: 10710 Error: 0 Ok: 10710 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:52:15,184 | Exp. Rate 240 req/sec Sent: 11090 Received: 11090 Error: 0 Ok: 11090 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:53:15,158 | Exp. Rate 250 req/sec Sent: 14420 Received: 14410 Error: 0 Ok: 14410 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:54:15,170 | Exp. Rate 260 req/sec Sent: 14500 Received: 14510 Error: 0 Ok: 14510 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:55:15,173 | Exp. Rate 270 req/sec Sent: 14220 Received: 14220 Error: 0 Ok: 14220 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:56:15,171 | Exp. Rate 280 req/sec Sent: 14400 Received: 14400 Error: 0 Ok: 14400 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:57:15,177 | Exp. Rate 290 req/sec Sent: 14240 Received: 14240 Error: 0 Ok: 14240 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:58:15,176 | Exp. Rate 300 req/sec Sent: 13790 Received: 13790 Error: 0 Ok: 13790 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 21:59:15,159 | Exp. Rate 310 req/sec Sent: 14790 Received: 14780 Error: 0 Ok: 14780 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 22:00:15,174 | Exp. Rate 320 req/sec Sent: 16070 Received: 16080 Error: 0 Ok: 16080 Exceeded: 10 Cancelled: 0
  INFO | 2021-03-24 22:01:15,193 | Exp. Rate 330 req/sec Sent: 15920 Received: 15920 Error: 0 Ok: 15920 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 22:02:15,197 | Exp. Rate 340 req/sec Sent: 16660 Received: 16660 Error: 0 Ok: 16660 Exceeded: 2 Cancelled: 0
  INFO | 2021-03-24 22:03:15,207 | Exp. Rate 350 req/sec Sent: 16390 Received: 16390 Error: 0 Ok: 16390 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 22:04:15,228 | Exp. Rate 360 req/sec Sent: 16700 Received: 16700 Error: 0 Ok: 16700 Exceeded: 2 Cancelled: 0
  INFO | 2021-03-24 22:05:15,246 | Exp. Rate 370 req/sec Sent: 16400 Received: 16400 Error: 0 Ok: 16400 Exceeded: 2 Cancelled: 0
  INFO | 2021-03-24 22:06:15,250 | Exp. Rate 380 req/sec Sent: 18370 Received: 18370 Error: 0 Ok: 18370 Exceeded: 1 Cancelled: 0
  INFO | 2021-03-24 22:07:15,252 | Exp. Rate 390 req/sec Sent: 17060 Received: 17060 Error: 0 Ok: 17060 Exceeded: 2 Cancelled: 0
  INFO | 2021-03-24 22:08:15,276 | Exp. Rate 400 req/sec Sent: 20120 Received: 20120 Error: 0 Ok: 20120 Exceeded: 50 Cancelled: 0
  INFO | 2021-03-24 22:09:15,278 | Exp. Rate 410 req/sec Sent: 23010 Received: 23010 Error: 0 Ok: 23010 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 22:10:15,273 | Exp. Rate 420 req/sec Sent: 22910 Received: 22910 Error: 0 Ok: 22910 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 22:11:15,267 | Exp. Rate 430 req/sec Sent: 23250 Received: 23250 Error: 0 Ok: 23250 Exceeded: 1 Cancelled: 0
  INFO | 2021-03-24 22:12:15,262 | Exp. Rate 440 req/sec Sent: 22760 Received: 22760 Error: 0 Ok: 22760 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 22:13:15,262 | Exp. Rate 450 req/sec Sent: 22980 Received: 22980 Error: 0 Ok: 22980 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 22:14:15,260 | Exp. Rate 460 req/sec Sent: 23900 Received: 23900 Error: 0 Ok: 23900 Exceeded: 0 Cancelled: 0
  INFO | 2021-03-24 22:15:15,252 | Exp. Rate 470 req/sec Sent: 24830 Received: 24830 Error: 0 Ok: 24830 Exceeded: 0 Cancelled: 0
  </code></pre>