title: Gateway Timeout 504 on multiple requests. Apache
body: <p>I have an XML file localy. It contains data from marketplace.
  It roughly looks like this:</p>
  
  <pre><code>&lt;offer id="2113"&gt;
      &lt;picture&gt;https://anotherserver.com/image1.jpg&lt;/picture&gt;
      &lt;picture&gt;https://anotherserver.com/image2.jpg&lt;/picture&gt;
  &lt;/offer&gt;
  &lt;offer id="2117"&gt;
      &lt;picture&gt;https://anotherserver.com/image3.jpg&lt;/picture&gt;
      &lt;picture&gt;https://anotherserver.com/image4.jpg&lt;/picture&gt;
  &lt;/offer&gt;
  ...
  </code></pre>
  
  <p>What I want is to save those images in <code>&lt;picture&gt;</code> node localy.</p>
  
  <p>There are about <em>9,000 offers</em> and about <em>14,000 images.</em></p>
  
  <p>When I iterate through them I see that images are being copied from that another server but at some point it gives <code>504 Gateway Timeout</code>.</p>
  
  <p>Thing is that sometimes error is given after 2,000 images sometimes way more or less.</p>
  
  <p>I tried getting only one image 12,000 times from that server (i.e. only <code>https://anotherserver.com/image3.jpg</code>) but it still gave the same error.</p>
  
  <p>As I've read, than another server is blocking my requests after some quantity. </p>
  
  <p>I tried using <code>PHP sleep(20)</code> after every 100th image but it still gave me the same error (<code>sleep(180) - same</code>). When I tried local image but with full path it didn't gave any errors. Tried second server (non local) the same thing occured.</p>
  
  <p>I use <code>PHP copy()</code> function to move image from that server. 
  I've just used <code>file_get_contents()</code> for testing purposes but got the same error. </p>
  
  <p>I have </p>
  
  <pre><code>set_time_limit(300000);
  ini_set('default_socket_timeout', 300000);
  </code></pre>
  
  <p>as well but no luck.</p>
  
  <p>Is there any way to do this without chunking requests?</p>
  
  <p>Does this error occur on some one image? Would be great to catch this error or just keep track of the response delay to send another request after some time if this can be done?</p>
  
  <p>Is there any constant time in seconds that I have to wait in order to get those requests rollin'?</p>
  
  <p>And pls give me non-curl answers if possible.</p>
  
  <blockquote>
    <p><strong>UPDATE</strong></p>
  </blockquote>
  
  <p>Curl and exec(wget) didn't work as well. They both gone to same error.</p>
  
  <p>Can remote server be tweaked so it doesn't block me? (If it does).</p>
  
  <p>p.s. if I do: <code>echo "&lt;img src = 'https://anotherserver.com/image1.jpg'" /&gt;</code> in loop for all 12,000 images, they show up just fine.</p>