title: Parallel.ForEach vs Async Forloop in Heavy I/O Ops
body: <p>I want to compare two theoretical scenarios. I have simplified the cases for purpose of the question. But basically its your typical producer consumer scenario. (I'm focusing on the consumer).</p>
  
  <p>I have a large <code>Queue&lt;string&gt; dataQueue</code> that I have to transmit to multiple clients.</p>
  
  <p>So lets start with the simpler case:</p>
  
  <pre><code> class SequentialBlockingCase
   {
      public static Queue&lt;string&gt; DataQueue = new Queue&lt;string&gt;();
      private static List&lt;string&gt; _destinations = new List&lt;string&gt;();
  
      /// &lt;summary&gt;
      /// Is the main function that is run in its own thread
      /// &lt;/summary&gt;
      private static void Run()
      {
          while (true)
          {
              if (DataQueue.Count &gt; 0)
              {
                  string data = DataQueue.Dequeue();
                  foreach (var destination in _destinations)
                  {
                      SendDataToDestination(destination, data);
                  }
              }
              else
              {
                  Thread.Sleep(1);
              }
          }
      }
  
      private static void SendDataToDestination(string destination, string data)
      {
          //TODO: Send data using http post, instead simulate the send
          Thread.Sleep(200);
      }
  }
  }
  </code></pre>
  
  <p>Now this setup works just fine. It sits there and polls the <code>Queue</code> and when there is data to send it sends it to all the destinations.</p>
  
  <p>Issues:</p>
  
  <ul>
  <li>If one of the destinations is unavailable or slow, it effects all of the other destinations.</li>
  <li>It does not make use of multi threading in the case of parallel execution.</li>
  <li>Blocks for every transmission to each destination.</li>
  </ul>
  
  <p>So here is my second attempt:</p>
  
  <pre><code> class ParalleBlockingCase
  {
      public static Queue&lt;string&gt; DataQueue = new Queue&lt;string&gt;();
      private static List&lt;string&gt; _destinations = new List&lt;string&gt;();
  
      /// &lt;summary&gt;
      /// Is the main function that is run in its own thread
      /// &lt;/summary&gt;
      private static void Run()
      {
          while (true)
          {
              if (DataQueue.Count &gt; 0)
              {
                  string data = DataQueue.Dequeue();
                  Parallel.ForEach(_destinations, destination =&gt;
                  {
                      SendDataToDestination(destination, data);
                  });
              }
              else
              {
                  Thread.Sleep(1);
              }
          }
      }
  
      private static void SendDataToDestination(string destination, string data)
      {
          //TODO: Send data using http post
          Thread.Sleep(200);
      }
  }
  </code></pre>
  
  <p>This revision at least does not effect the other destinations if 1 destination is slow or unavailable.</p>
  
  <p>However this method is still blocking and I am not sure if <code>Parallel.ForEach</code> makes use of the thread pool. My understanding is that it will create X number of threads / tasks and execute 4 (4 core cpu) at a time. But it has to completely Finnish task 1 before task 5 can start.</p>
  
  <p>Hence My 3rd option:</p>
  
  <pre><code>class ParalleAsyncCase
  {
      public static Queue&lt;string&gt; DataQueue = new Queue&lt;string&gt;();
      private static List&lt;string&gt; _destinations = new List&lt;string&gt; { };
  
      /// &lt;summary&gt;
      /// Is the main function that is run in its own thread
      /// &lt;/summary&gt;
      private static void Run()
      {
          while (true)
          {
              if (DataQueue.Count &gt; 0)
              {
                  string data = DataQueue.Dequeue();
                  List&lt;Task&gt; tasks = new List&lt;Task&gt;();
                  foreach (var destination in _destinations)
                  {
                      var task = SendDataToDestination(destination, data);
                      task.Start();
                      tasks.Add(task);
                  }
  
                  //Wait for all tasks to complete
                  Task.WaitAll(tasks.ToArray());
              }
              else
              {
                  Thread.Sleep(1);
              }
          }
      }
  
      private static async Task SendDataToDestination(string destination, string data)
      {
          //TODO: Send data using http post
          await Task.Delay(200);
      }
  }
  </code></pre>
  
  <p>Now from my understanding this option, will still block on the main thread at <code>Task.WaitAll(tasks.ToArray());</code> which is fine because I don't want it to run away with creating tasks faster than they can be executed.</p>
  
  <p>But the tasks that will be execute in parallel should make use of the <code>ThreadPool</code>, and all X number of tasks should start executing at once, not blocking or in sequential order.  (thread pool will swap between them as they become active or are <code>awaiting</code>)</p>
  
  <p>Now my question.</p>
  
  <p>Does option 3 have any performance benefit over option 2.</p>
  
  <p>Specifically in a higher performance server side scenario. In the specific software I am working on now. There would be multiple instanced of my simple use case above. Ie several consumers.</p>
  
  <p>I'm interested in the theoretical differences and pro's vs cons of the two solutions, and maybe even a better 4th option if there is one.</p>