title: How to Loop calls to Pagination URL in C# HttpClient to download all Pages from JSON results
body: <p>My 1st question, so please be kind... :)</p>
  
  <p>I'm using the <strong>C#</strong> <code>HttpClient</code> to invoke Jobs API Endpoint. </p>
  
  <p>Here's the endpoint: <a href="http://service.dice.com/api/rest/jobsearch/v1/simple.json?skill=ruby" rel="noreferrer">Jobs API Endpoint (doesn't require key, you can click it)</a></p>
  
  <p>This gives me JSON like so. </p>
  
  <pre><code>{
    "count": 1117,
    "firstDocument": 1,
    "lastDocument": 50,
    "nextUrl": "\/api\/rest\/jobsearch\/v1\/simple.json?areacode=&amp;country=&amp;state=&amp;skill=ruby&amp;city=&amp;text=&amp;ip=&amp;diceid=&amp;page=2",
    "resultItemList": [
      {
        "detailUrl": "http:\/\/www.dice.com\/job\/result\/90887031\/918715?src=19",
        "jobTitle": "Sr Security Engineer",
        "company": "Accelon Inc",
        "location": "San Francisco, CA",
        "date": "2017-03-30"
      },
      {
        "detailUrl": "http:\/\/www.dice.com\/job\/result\/cybercod\/BB7-13647094?src=19",
        "jobTitle": "Platform Engineer - Ruby on Rails, AWS",
        "company": "CyberCoders",
        "location": "New York, NY",
        "date": "2017-04-16"
      }
   ]
  }
  </code></pre>
  
  <p>I've pasted a complete JSON snippet so you can use it in your answer. The full results are really long for here.</p>
  
  <p>Here's are the C# classes. </p>
  
  <pre><code>using Newtonsoft.Json;
  using System.Collections.Generic;
  
  namespace MyNameSpace
  {
      public class DiceApiJobWrapper
      {
          public int count { get; set; }
          public int firstDocument { get; set; }
          public int lastDocument { get; set; }
          public string nextUrl { get; set; }
  
          [JsonProperty("resultItemList")]
          public List&lt;DiceApiJob&gt; DiceApiJobs { get; set; }
      }
  
      public class DiceApiJob
      {
          public string detailUrl { get; set; }
          public string jobTitle { get; set; }
          public string company { get; set; }
          public string location { get; set; }
          public string date { get; set; }
      }
  }
  </code></pre>
  
  <p>When I invoke the URL using HttpClient and deserialize using JSON.NET, I do get the data back properly. </p>
  
  <p>Here's the code I am calling from my Console App's <code>Main</code> method (hence the <code>static</code> list, I think this could be better refactored??)</p>
  
  <pre><code>   private static List&lt;DiceApiJob&gt; GetDiceJobs()
      {
          HttpClient httpClient = new HttpClient();
          var jobs = new List&lt;DiceApiJob&gt;();
  
          var task = httpClient.GetAsync("http://service.dice.com/api/rest/jobsearch/v1/simple.json?skill=ruby")
            .ContinueWith((taskwithresponse) =&gt;
            {
                var response = taskwithresponse.Result;
                var jsonString = response.Content.ReadAsStringAsync();
                jsonString.Wait();
  
                var result =  JsonConvert.DeserializeObject&lt;DiceApiJobWrapper&gt;(jsonString.Result);
                if (result != null)
                {
                    if (result.DiceApiJobs.Any())
                        jobs = result.DiceApiJobs.ToList();
  
                    if (result.nextUrl != null)
                    {
                        //
                        // do this GetDiceJobs again in a loop? How?? Any other efficient elegant way??
                    }
                }
            });
          task.Wait();
  
          return jobs;
      }
  </code></pre>
  
  <p>But now, how do I check if there are more jobs using the <code>nextUrl</code> field? I know I can check to see if it's not null, and if if not, that means there are more jobs to pull down. </p>
  
  <p><a href="https://i.stack.imgur.com/N3sn7.png" rel="noreferrer">Results from my debugging and stepping through</a></p>
  
  <p>How do I do this recursively, and without hanging and with some delays so I don't cross the API limits? I think I have to use TPL ( Task Parallel Library) but am quite baffled. </p>
  
  <p>Thank you!
  ~Sean</p>