title: How to dump all results of a API request when there is a page limit?
body: <p>I am using an API to pull data from a url, however the API has a pagination limit. It goes like:</p>
  <ul>
  <li>Page (default is 1 and it's the page number you want to retrieve)</li>
  <li>Per_page (default is 100 and it's the maximum number of results returned in the response(max=500))</li>
  </ul>
  <p>I have a script which I can get the results of a page or per page but I want to automate it. I want to be able to loop through all the pages or per_page(500) and load it in to a json file.</p>
  <p>Here is my code that can get 500 results per_page:</p>
  <pre><code>import json, pprint
  import requests
  
  url = &quot;https://my_api.com/v1/users?per_page=500&quot;
  header = {&quot;Authorization&quot;: &quot;Bearer &lt;my_api_token&gt;&quot;}
  
  s = requests.Session()
  s.proxies = {&quot;http&quot;: &quot;&lt;my_proxies&gt;&quot;, &quot;https&quot;: &quot;&lt;my_proxies&gt;&quot; }
  
  resp = s.get(url, headers=header, verify=False)
  raw=resp.json()
  for x in raw:
      print(x)
  </code></pre>
  <p>The output is 500 but is there a way to keep going and pull the results starting from where it left off? Or even go by page and get all the data per page until there's no data in a page?</p>