title: Retrieving https page using .net HttpWebRequest or WebClient returns (400) Bad Request
body: <p>I have an account with Interactive Brokers. They have a decent trading API, but there is no API to access some of the account management features that are available through their web site at <a href="https://www.interactivebrokers.com/sso/Login" rel="nofollow noreferrer">https://www.interactivebrokers.com/sso/Login</a> . I looked at the javascript a bit, and they are encrypting the user name &amp; password in the browser in javascript, and then sending the encrypted stuff over SSL.</p>
  
  <p>I'm hoping to be able to do that in c#. The first step (I think) is to open <a href="https://www.interactivebrokers.com/sso/Login" rel="nofollow noreferrer">https://www.interactivebrokers.com/sso/Login</a> using an HttpWebRequest or WebClient, to get a session ID (this is just loading the login page -- not even entering any username or password yet). </p>
  
  <p>There's nothing too ground-breaking in the code. The WebClient version is:</p>
  
  <pre><code>WebClient client = new WebClient();
  client.Encoding = Encoding.UTF8;
  Stream data = client.OpenRead("https://www.interactivebrokers.com/sso/Login");
  </code></pre>
  
  <p>The HttpWebRequest version is nothing to write home about either:</p>
  
  <pre><code>HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://www.interactivebrokers.com/sso/Login");
  HttpWebResponse response = (HttpWebResponse)request.GetResponse();
  </code></pre>
  
  <p>However, in either case when I do that, I get a </p>
  
  <pre><code>The remote server returned an error: (400) Bad Request.
  </code></pre>
  
  <p>I tried to add a user agent, as per <a href="https://stackoverflow.com/questions/702079/c-simple-httpwebrequest-question">Why Does my HttpWebRequest Return 400 Bad request?</a> :</p>
  
  <pre><code>request.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)";
  </code></pre>
  
  <p>and</p>
  
  <pre><code>request.UserAgent = "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET4.0C; .NET4.0E)";
  </code></pre>
  
  <p>I still get the (400) error.</p>
  
  <p>The inner exception is null, but looking at the WebException's stream shows the following response (I don't know why the title says '200 OK'):</p>
  
  <pre><code>&lt;!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"&gt;
  &lt;html&gt;&lt;head&gt;
  &lt;title&gt;200 OK&lt;/title&gt;
  &lt;/head&gt;&lt;body&gt;
  &lt;h1&gt;OK&lt;/h1&gt;
  &lt;p&gt;Your browser sent a request that this server could not understand.&lt;br /&gt;
  &lt;/p&gt;
  &lt;/body&gt;&lt;/html&gt;
  </code></pre>
  
  <p>I looked at the difference in packets between a web browser doing the request and my c# code, and the first difference I noticed is that the web browser has many more cypher suites available to it then my c# code. When the server responds to the web browser, it chooses one of the cyphers that is not available in the c# client. How do I add cypher suites to HttpWebRequest / WebClient? I haven't yet figured out if that is actually the problem or not.</p>
  
  <p>Am I missing something really basic? Why can't I even load that page with an HttpWebRequest or WebClient?</p>
  
  <p>Thanks</p>