title: TcpListener is queuing connections faster than I can clear them
body: <p>As I understand it, <code>TcpListener</code> will queue connections once you call <code>Start()</code>. Each time you call <code>AcceptTcpClient</code> (or <code>BeginAcceptTcpClient</code>), it will dequeue one item from the queue.</p>
  
  <p>If we load test our <code>TcpListener</code> app by sending 1,000 connections to it at once, the queue builds far faster than we can clear it, leading (eventually) to timeouts from the client because it didn't get a response because its connection was still in the queue. However, the server doesn't appear to be under much pressure, our app isn't consuming much CPU time and the other monitored resources on the machine aren't breaking a sweat. It feels like we're not running efficiently enough right now.</p>
  
  <p>We're calling <code>BeginAcceptTcpListener</code> and then immediately handing over to a <code>ThreadPool</code> thread to actually do the work, then calling <code>BeginAcceptTcpClient</code> again. The work involved doesn't seem to put any pressure on the machine, it's basically just a 3 second sleep followed by a dictionary lookup and then a 100 byte write to the <code>TcpClient</code>'s stream. </p>
  
  <p>Here's the <code>TcpListener</code> code we're using:</p>
  
  <pre><code>    // Thread signal.
      private static ManualResetEvent tcpClientConnected = new ManualResetEvent(false);
  
      public void DoBeginAcceptTcpClient(TcpListener listener)
      {
          // Set the event to nonsignaled state.
          tcpClientConnected.Reset();
  
          listener.BeginAcceptTcpClient(
              new AsyncCallback(DoAcceptTcpClientCallback),
              listener);
  
          // Wait for signal
          tcpClientConnected.WaitOne();
      }
  
      public void DoAcceptTcpClientCallback(IAsyncResult ar)
      {
          // Get the listener that handles the client request, and the TcpClient
          TcpListener listener = (TcpListener)ar.AsyncState;
          TcpClient client = listener.EndAcceptTcpClient(ar);
  
          if (inProduction)
              ThreadPool.QueueUserWorkItem(state =&gt; HandleTcpRequest(client, serverCertificate));  // With SSL
          else
              ThreadPool.QueueUserWorkItem(state =&gt; HandleTcpRequest(client));  // Without SSL
  
          // Signal the calling thread to continue.
          tcpClientConnected.Set();
      }
  
      public void Start()
      {
          currentHandledRequests = 0;
          tcpListener = new TcpListener(IPAddress.Any, 10000);
          try
          {
              tcpListener.Start();
  
              while (true)
                  DoBeginAcceptTcpClient(tcpListener);
          }
          catch (SocketException)
          {
              // The TcpListener is shutting down, exit gracefully
              CheckBuffer();
              return;
          }
      }
  </code></pre>
  
  <p>I'm assuming the answer will be related to using <code>Sockets</code> instead of <code>TcpListener</code>, or at least using <code>TcpListener.AcceptSocket</code>, but I wondered how we'd go about doing that?</p>
  
  <p>One idea we had was to call <code>AcceptTcpClient</code> and immediately <code>Enqueue</code> the <code>TcpClient</code> into one of multiple <code>Queue&lt;TcpClient&gt;</code> objects. That way, we could poll those queues on separate threads (one queue per thread), without running into monitors that might block the thread while waiting for other <code>Dequeue</code> operations. Each queue thread could then use <code>ThreadPool.QueueUserWorkItem</code> to have the work done in a <code>ThreadPool</code> thread and then move onto dequeuing the next <code>TcpClient</code> in its queue. Would you recommend this approach, or is our problem that we're using <code>TcpListener</code> and no amount of rapid dequeueing is going to fix that?</p>