title: Selenium headless browser webdriver [Errno 104] Connection reset by peer
body: <p>I am trying to scrape data from the URLs below. But selenium fails when <code>driver.get(url)</code> Some times the error is <code>[Errno 104] Connection reset by peer</code>, sometimes <code>[Errno 111] Connection refused</code>. <strong>On rare days it works just fine</strong> and on my mac with real browser the same spider works fine every single time. So this isn't related to my <code>spider</code>.</p>
  
  <p>Have tried many solutions like waiting got selectors on page, implicit wait, using selenium-requests yo pass proper request headers, etc. But nothing seems to work. </p>
  
  <pre><code>http://www.snapdeal.com/offers/deal-of-the-day
  https://paytm.com/shop/g/paytm-home/exclusive-discount-deals
  </code></pre>
  
  <p>I am using <code>python</code>, <code>selenium</code> &amp; <code>headless Firefox webdriver</code> to achieve this. The os is <code>centos 6.5</code>. </p>
  
  <p><em>Note:</em> I have many <code>AJAX</code> heavy pages that gets scraped successfully some are below.</p>
  
  <pre><code>http://www.infibeam.com/deal-of-the-day.html, http://www.amazon.in/gp/goldbox/ref=nav_topnav_deals
  </code></pre>
  
  <p>Already spent many days trying to debug the issue with no luck. Any help would be appreciated.</p>