title: Multiple parallel awaitable tasks
body: <p>I'm experimenting with ways to improve performance in our ASP.NET applications. One of the things I'm looking at is using parallelism and making operations async to try to reduce processing time and improve throughput. I started by mocking up something we do fairly frequently, issue multiple database retrievals to render a page.</p>
  
  <pre><code>public ActionResult Index()
  {
      var dal = new Dal();
      var cases = new List&lt;Case&gt;();
      cases.AddRange( dal.GetAssignedCases() );
      cases.AddRange( dal.GetNewCases() );
      return View( "Cases", cases );
  }
  </code></pre>
  
  <p>The two Dal methods use <code>Thread.Sleep(2000)</code> to simulate a query and just return a collection of hard-coded objects. I run this with Apache Bench using <code>ab -c 1 -n 1</code> and it takes about four seconds. My first attempt to try to improve it was:</p>
  
  <pre><code>public ActionResult Index()
  {
      var dal = new Dal();
      var assignedCases = Task.Factory.StartNew( () =&gt; dal.GetAssignedCases() );
      var newCases = Task.Factory.StartNew( () =&gt; dal.GetNewCases() );
      IEnumerable&lt;Case&gt;[] allCases = Task.WhenAll( assignedCases, newCases ).Result;
      return View( "Cases", allCases.SelectMany( c =&gt; c ) );
  }
  </code></pre>
  
  <p>When I run this using the same <code>ab</code> command it shows about two seconds, which makes sense because I'm running two tasks each of which takes two seconds but they're running in parallel.</p>
  
  <p>When I change the benchmark to 10 concurrent requests (i.e. <code>ab -n 10 -c 10</code>) I get the following.</p>
  
  <pre><code>Fulfilled  Original Parallel
   50%         4014     2038
   66%         4015     2039
   75%         4017     4011
  </code></pre>
  
  <p>The rest of the numbers up to 100% are similar in both columns.</p>
  
  <p>I'm assuming that what I'm running into here is thread pool contention. About 2/3 of the requests are fulfilled quickly and after that stuff is sitting around waiting for threads to service the requests. So I think maybe if I added async to the mix I could get even more requests being served more quickly. That's where I start having problems and I don't know if the problem is the way I'm simulating long-running queries or the way I'm using the language feature or if I'm just completely on the wrong track and the light at the end of the tunnel is an on-coming train. :-)</p>
  
  <p>First thing I did was create a DalAsync. In DalAsync I replaced the <code>Thread.Sleep(2000)</code> with <code>await Task.Delay(2000)</code>, marked each method with the <code>async</code> keyword and changed the return type from <code>IEnumerable&lt;Case&gt;</code> to <code>Task&lt;IEnumerable&lt;Case&gt;&gt;</code>. I then wrote a new controller method pieced together from information I've read in a half-dozen blog posts and MSDN articles.</p>
  
  <pre><code>public async Task&lt;ActionResult&gt; Index()
  {
      var dal = new DalAsync();
      var assignedCases = dal.GetAssignedCasesAsync();
      var newCases = dal.GetNewCasesAsync();
      var allCases = await Task.WhenAll( assignedCases, newCases );
      return View( "Cases", allCases.SelectMany( c =&gt; c ) );
  }
  </code></pre>
  
  <p>When I run this using <code>ab</code> it never finishes, even with one request it ends up timing out. I also tried the following variation, which works but returns numbers almost identical to the original version (which sort of makes sense, because it seems I'm serializing the queries again).</p>
  
  <pre><code>var assignedCases = await dal.GetAssignedCasesAsync();
  var newCases = await dal.GetNewCasesAsync();
  var allCases = new List&lt;Case&gt;( assignedCases );
  allCases.AddRange( newCases );
  </code></pre>
  
  <p>What I'd like to have happen is:</p>
  
  <ul>
  <li>Run the two queries in parallel</li>
  <li>When the controller is waiting for the Dal methods to respond it frees up the thread and lets other
  requests execute.</li>
  </ul>