title: Scraping fanduel sportsbook for odds with Beautiful Soup
body: <p>So i'm very new to web scraping with python and I have a question regarding a project i'm working on. I'm attempting to implement a betting strategy based off of something called the kelly criterion. In a nutshell, it asserts that there is an optimal fixed fraction (f) of your starting bankroll to bet based on the expected value (EV) of a bet in order to maximize growth.</p>
  
  <p>EV = (b<em>p - a</em>q)</p>
  
  <p>f =  EV / [b*a]</p>
  
  <p>Optimal_Bet = f*C</p>
  
  <p>Where, </p>
  
  <p>b = potential units won</p>
  
  <p>a = potential units lost (while accounting for "vigorish")</p>
  
  <p>p = Team 1 win probability</p>
  
  <p>q = Team 2 win probability</p>
  
  <p>C = Initial bankroll before bet</p>
  
  <p>b,a are based off of the odds that the sportsbook offers. Approximating p,q of a given team is a separate challenge.</p>
  
  <p>I already have a function that takes in C and the odds in American format of a particular matchup (ex. +102 &amp; -120) and calculates b,a,EV,f, and the Optimal_Bet. 
  I would like to scrape the sportsbook I use (fanduel) for all the betting lines in a particular sport, and feed them into the function in real time in order to spit out EV,f, and the corresponding optimal bet for each matchup.</p>
  
  <p>Finally, my problem is that when I request the content from the website and create a soup object, it doesn't seem to be getting everything out of the html document. </p>
  
  <p>Here is my python code:</p>
  
  <pre><code>import requests
  from bs4 import BeautifulSoup
  
  fanduel_nfl_response = requests.get('https://sportsbook.fanduel.com/sports/navigation/6227.1/11147.3')
  fanduel_nba_response = requests.get('https://sportsbook.fanduel.com/sports/navigation/830.1/8047.1')
  fanduel_cbb_response = requests.get('https://sportsbook.fanduel.com/sports/navigation/11086.3/11087.3')
  
  fanduel_nfl_page = fanduel_nfl_response.content
  fanduel_nba_page = fanduel_nba_response.content
  fanduel_cbb_page = fanduel_cbb_response.content
  
  nfl_soup = BeautifulSoup(fanduel_nfl_page,"lxml")
  nba_soup = BeautifulSoup(fanduel_nba_page, "lxml")
  cbb_soup = BeautifulSoup(fanduel_cbb_page, "lxml")
  
  print(nba_soup)
  </code></pre>
  
  <p>When I print either of the three soup objects, I get an output that does not seem to contain any of the information I want, even though i'm certain the website links I chose are correct. So I can't use .find_all() to search for anything because it's just not there. <a href="https://i.stack.imgur.com/oo0dI.jpg" rel="nofollow noreferrer">Here is a picture of what the nba line page looks like when I inspect it in chrome. The line highlighted in blue as well as the five lines below it contain the information that I want.</a></p>
  
  <p><a href="https://i.stack.imgur.com/OZOsb.jpg" rel="nofollow noreferrer">Here is a picture of the output to my code</a></p>
  
  <p>Is there a reason why my content request isn't getting the whole html document, or am I missing something entirely? Am I actually getting all the content with my request and the stuff I want is just not visible? I am very new to this so any responses would be greatly appreciated. Thanks!</p>