title: Extract data from a paginated API with tRESTClient
body: <p>I need to crawl a paginated REST API with <code>offset</code> and <code>limit</code> parameters using <a href="http://www.talend.com" rel="nofollow noreferrer">Talend</a>.
  The API gives me a list of the resources I am interested in.</p>
  
  <p>For instance, the response to the initial request with <code>offset=0</code> and <code>limit=2</code> is:</p>
  
  <pre><code>{
  
      "meta": {
          "limit": 2,
          "next": "/api/v1/request/?offset=2&amp;limit=2",
          "offset": 0,
          "previous": null,
          "total_count": 4300
      },
      "objects": [
          {
              "id": 1,
              "name": "foo"
          },
          {
              "id": 2,
              "name": "bar"
          }
      ]
  }
  </code></pre>
  
  <p>As you can see, the response object contains an <code>objects</code> key, i.e. some of the desired resources and a <code>meta</code> key which indicates the next URL to query: <code>next</code>. So far I am able to perform the initial request with <code>tRESTClient</code>. However, I don't know how to proceed from here and request the remaining pages using the clue given by <code>next</code>.</p>
  
  <p><strong>How can I</strong> perform multiple requests to that API so that I <strong>iterate over the whole list until <code>next</code> equals <code>null</code> (=list is exhausted)?</strong></p>
  
  <p>I tried to figure out how tSetGlobalVar and tLoop could help me, but so far with no success. But then again, I am a Talend newbie.</p>
  
  <h2>Current job</h2>
  
  <p>This is what my job currently looks like:</p>
  
  <p><img src="https://i.stack.imgur.com/LuiAg.png" alt="Screenshot of the current job"></p>