title: urllib: Opening a url always gets 429: Too many requests
body: <p>I just got started with the <code>urllib</code> module. I'm trying to scrape products from supermarkets and there's a website that seems to <strong>always</strong> respond with an <code>HTTP Error 429: Too many requests</code>. I already did a bit of research on the Stack Overflow and no one seems to have the same problem. My code is as simple as it can get:</p>
  
  <pre><code>&gt;&gt;&gt; import urllib.request
  &gt;&gt;&gt; resp = urllib.request.urlopen("https://shop.coles.com.au/a/a-national/product/head-shoulders-shampoo-conditioner-2in1-deep-clean")
  Traceback (most recent call last):
    File "&lt;pyshell#1&gt;", line 1, in &lt;module&gt;
      resp = urllib.request.urlopen("https://shop.coles.com.au/a/a-national/product/head-shoulders-shampoo-conditioner-2in1-deep-clean")
    File "C:\Users\thank\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 222, in urlopen
  return opener.open(url, data, timeout)
    File "C:\Users\thank\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 531, in open
  response = meth(req, response)
    File "C:\Users\thank\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 640, in http_response
  'http', request, response, code, msg, hdrs)
    File "C:\Users\thank\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 568, in error
  return self._call_chain(*args)
    File "C:\Users\thank\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 503, in _call_chain
  result = func(*args)
    File "C:\Users\thank\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 648, in http_error_default
  raise HTTPError(req.full_url, code, msg, hdrs, fp)
  urllib.error.HTTPError: HTTP Error 429: Too Many Requests
  </code></pre>
  
  <p>I've also tried to modify the user-agent as <a href="https://stackoverflow.com/a/13214012">this answer</a> suggests, but the result is still the same</p>
  
  <p>Can someone explain which default settings inside the urllib module may cause the problem? Or is it because the website blocks bots? Other product pages of the website don't work either.</p>