title: Python sockets and UDP: how the packet queue helps with a sender that's faster than a receiver
body: <p>There's a receiver that loops and calls <code>socket.recv</code>:</p>
  
  <pre><code>import socket
  import time
  
  UDP_IP = "127.0.0.1"
  UDP_PORT = 5005
  
  sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)  # Internet  # UDP
  
  sock.bind((UDP_IP, UDP_PORT))
  
  sock.settimeout(0.001)
  
  while True:
      try:
          data = sock.recv(1024)  # buffer size is 1024 bytes
      except socket.timeout:
          pass
  
      time.sleep(0.01)
  </code></pre>
  
  <hr>
  
  <p>And a sender loops and sends messages with <code>socket.sendto</code>, counting upward:</p>
  
  <pre class="lang-py prettyprint-override"><code>import socket
  
  UDP_IP = "127.0.0.1"
  UDP_PORT = 5005
  
  sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)  # Internet  # UDP
  
  TOTAL_TO_SEND = 10000
  
  messages = [str.encode(f"{num}") for num in range(TOTAL_TO_SEND)]
  
  for total_sent in messages:
      sock.sendto(total_sent, (UDP_IP, UDP_PORT))
  </code></pre>
  
  <hr>
  
  <p>With this setup, the receiver is not looping fast enough to get back to the <code>recv</code> line by the time the sender has sent the next message.</p>
  
  <p>With UDP allowing lost data, I thought perhaps the messages that arrived while the receiver hadn't gotten back to <code>recv</code> yet would simply be skipped. But that's not the case.</p>
  
  <p>When I send only 10,000 messages (<code>TOTAL_TO_SEND = 10000</code>), even though the send finishes almost instantly, the receiver takes some time to get through all 10000 messages, but it doesn't skip any. This implies there is some buffer/queue for messages that arrived while the receiver wasn't ready, and they're being stored until the receiver code is able to handle them all.</p>
  
  <p>But when I send 10x as many messages (<code>TOTAL_TO_SEND = 100000</code>), there is some cutoff (15355 for my latest run) below which every single message is handled, and after which it skips hundreds at a time. This implies to me that the buffer/queue became full, and the loop was then only fast enough to handle one message out of every couple hundred sent.</p>
  
  <p>More evidence of the finite-size-queue theory is that increasing the message size (by prepending some 0's to each message) lowers the cutoff where messages start getting skipped. Prepending 100 0's to each message causes the cutoff to be at 641 instead of 15355.</p>
  
  <p>I have not been able to find a thorough explanation of this behavior, so wondered if anyone here knew how this "packet queue" worked in Python sockets, and what its parameters are. Is its size known? Can it be increased? Is there a time limit before messages disappear? Is there documentation of this?</p>
  
  <p>Thanks!</p>