title: Error code 401 in Java program
body: <p>I am trying to crawl a website using a java program.
  Until last night it was working perfect, but now the server returns error code 401.</p>
  
  <p>HOWEVER, I can still see the pages that I want through my web browsers.
  So, I don't know what is wrong? If the server add my IP to black list, so why can I see the URLs through the web browsers? If not, what else can cause 401 error?</p>
  
  <p>Two more points:
  There is no username and password for this site and authentication is based on my IP. </p>
  
  <p>Also, I tried to change my user agent, and now I get Error 503.</p>