title: Asynchronous server design
body: <p>We have a server receiving data from between 500-1500 GPS devices. Each device sends a packet containing around 1-4 GPS coordinates every 10-30 seconds. The server is designed asynchronously with a listener handling connections using Begin- EndAccept, and communication using Begin- EndReceive. Once a packet is received the data is processed and stored in a database. </p>
  
  <p>With few devices (500-700 devices) this takes barely 50 ms, and we have less than 50 concurrent threads running, and a realistic CPU usage (20-40%). However when the server is pressured with connections (1000+) the number of threads explodes to 500-600 and the CPU usage also drops to a few %. The processing time is also increased to several minutes.</p>
  
  <p>Is the asynchronous design bad for this particular scenario with many small packets being sent at this rate, or might the be a problem in the code?</p>
  
  <p>We have currently had to distribute the load across three servers to accomodate all devices, and they are all VMs with 6 CPUs and 4GB memory hosted on a Hyper-V server.</p>
  
  <p>SOLUTION:</p>
  
  <p>The solution I found from the answers from people, was to immediately schedule it as a task using the .Net parallel library, as this is much smarter when scheduling threads across multiple cores:</p>
  
  <pre><code>void EndReceive(IAsyncResult res)
  {
     Task.Factory.StartNew((object o) =&gt; { HandleReceive(o as IAsyncResult); }, res, TaskCreationOptions.PreferFairness);
  }
  </code></pre>
  
  <p>Now the threads rarely exceed 50.</p>