title: How to download files with node-fetch
body: <p>I need help implementing a file downloader in nodejs.</p>
  <p>So i need to download over 25'000 files from a server. Im using <code>node-fetch</code> but i don't exactly know how to implement this. I tried using <code>Promise.allSettled()</code> but i also need a way to limit the amount of concurrent requests to the server otherwise i get rate-limited.</p>
  <p>This is my code so far:</p>
  <pre class="lang-js prettyprint-override"><code>const fetch = require('node-fetch')
  
  async function main () {
    const urls = [
      'https://www.example.com/foo.png',
      'https://www.example.com/bar.gif',
      'https://www.example.com/baz.jpg',
      ... many more (~25k)
    ]
  
    // how to save each file on the machine with same file name and extension?
    // how to limit the amount of concurrent requests to the server?
    const files = await Promise.allSettled(
      urls.map((url) =&gt; fetch(url))
    )
  }
  
  main()
  </code></pre>
  <p>So my questions are:</p>
  <ul>
  <li>How do i limit the amount of concurrent requests to the server? Can this be solved using a custom https agent with <code>node-fetch</code> and setting the <code>maxSockets</code> to something like 10?</li>
  <li>How do i check if the file exists on the server and if it does then download it on my machine with the same file name and extension?</li>
  </ul>
  <p>It would be very helpful if someone could show a small example code how i would implement such functionality.</p>
  <p>Thanks in advance.</p>