title: Python requests to access OAUTH website content - SNL Finance
body: <p>I've been banging my head up against the wall trying to retrieve content from the news source "SNL Finance". I have valid credentials, so in theory I should be able to programmatically access their news content.</p>
  
  <p>In short, I've tried executing the below script but with no success:</p>
  
  <pre><code>s = requests.Session()
  
  client_id = "..."
  client_secret = "..."
  token_url = "https://www.snl.com/SNL.Services.Security.Service/oauth/token"
  protected_url = "https://www.snl.com/web/client?auth=inherit#news/article?id=40666532&amp;KeyProductLinkType=14"
  
  request_data = {
      "client_id": client_id,
      "client_secret": client_secret,
      "scope": "https://www.snl.com",
      "grant_type": "refresh_token",
      "refresh_token": refresh_token
  }
  
  token_response = s.post(token_url, data=request_data)
  ### token response is in jwt format, including token_type, expires_in, scope, etc. ###
  token = json.loads(token_response.text)["access_token"].split('&gt;')[1].split('&lt;')[0]
  request_data["token"] = token
  
  article = s.post(protected_url, headers=request_data)
  </code></pre>
  
  <p>Sadly, this fails. I end up with a 200 response, but it appears to just be the login page (honestly not entirely sure what I'm looking at).</p>
  
  <p>For more background, I've included browser information as it populates throughout the authentication process:</p>
  
  <ol>
  <li><p>Attempt to visit protected url, redirected to the below url (omitted snl base):</p>
  
  <pre><code>/web/client?auth=inherit&amp;contextType=external&amp;username=string&amp;enablePersistentLogin=true&amp;OverrideRetryLimit=0&amp;SwitchGetToPostLimit=50000&amp;contextValue=%2Foam&amp;password=secure_string&amp;challenge_url=https%3A%2F%2Fwww.snl.com%2Fweb%2Fclient%3Fauth%3Dinherit&amp;request_id=-6149669210818920852&amp;authn_try_count=0&amp;locale=en_US&amp;resource_url=https%253A%252F%252Fwww.snl.com%252FInteractiveX%252FDefault.aspx%253Ftarget%253Dnews%25252Farticle%25253Fid%25253D40666532%252526KeyProductLinkType%25253D14%2526SNL3%253D1
  </code></pre></li>
  </ol>
  
  <p>Request headers are shown <a href="https://imgur.com/QJ9ZNDC" rel="nofollow noreferrer">here</a>.</p>
  
  <ol start="2">
  <li>Upon entering login / password, a token is retrieved and the protected page loads.</li>
  </ol>
  
  <p>Request cookies are as shown <a href="https://imgur.com/CWiMiPX" rel="nofollow noreferrer">here</a>.</p>
  
  <p>Also, I'm a bit confused at to why the token value of <code>SNL_OAUTH_TOKEN</code> in the above <a href="https://imgur.com/CWiMiPX" rel="nofollow noreferrer">link (second link)</a> differs from what is shown in the jwt token response I receive from my script.</p>
  
  <p>Any guidance here will be hugely appreciated. I'm also happy to send any other non-personal information that proves useful.</p>
  
  <p>Thank you!</p>