title: scraping 2 websites (same structure) with basically the same code but 2 different results?
body: <p>I scraped 2 websites:</p>
  <ol>
  <li><a href="https://www.brookings.edu/blog/future-development/2018/01/26/delivering-inclusive-urban-access-3-uncomfortable-truths/" rel="nofollow noreferrer">https://www.brookings.edu/blog/future-development/2018/01/26/delivering-inclusive-urban-access-3-uncomfortable-truths/</a>
  <a href="https://i.stack.imgur.com/uaE3m.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/uaE3m.png" alt="enter image description here" /></a></li>
  <li><a href="https://www.nytimes.com/2018/01/25/opinion/sunday/silicon-valley-immortality.html" rel="nofollow noreferrer">https://www.nytimes.com/2018/01/25/opinion/sunday/silicon-valley-immortality.html</a>
  <a href="https://i.stack.imgur.com/fSpL7.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/fSpL7.png" alt="enter image description here" /></a></li>
  </ol>
  <p>Before scraping, I created the following class and function:</p>
  <pre><code>class Content:
      def __init__(self, url, title, body):
          self.url = url
          self.title = title
          self.body = body
  def getPage(url):
      req = requests.get(url)
      return BeautifulSoup(req.text, 'html.parser')
  </code></pre>
  <p>For the first website I used following code:</p>
  <pre><code>url = 'https://www.brookings.edu/blog/future-development/2018/01/26/delivering-inclusive-urban-access-3-uncomfortable-truths/'
  
  bs = getPage(url)
  title = bs.find('h1').text
  body = bs.find('div', {'class': 'post-body'}).text
  utb = Content(url, title, body)
  
  print('Title: {}'.format(utb.title))
  print('URL: {}\n'.format(utb.url))
  print(utb.body)
  </code></pre>
  <p>Output:</p>
  <blockquote>
  <p>Title: Delivering inclusive urban access: 3 uncomfortable truths URL:
  <a href="https://www.brookings.edu/blog/future-development/2018/01/26/delivering-inclusive-urban-access-3-uncomfortable-truths/" rel="nofollow noreferrer">https://www.brookings.edu/blog/future-development/2018/01/26/delivering-inclusive-urban-access-3-uncomfortable-truths/</a></p>
  <p>The past few decades have been filled with a deep optimism about the
  role of cities and suburbs across the world. These engines of economic
  growth host a majority of world population, are major drivers of
  economic innovation, and have created pathways to opportunities for
  untold amounts of people.</p>
  <p>Jeffrey Gutman Nonresident Senior Fellow - Global Economy and
  Development</p>
  <p>Adie Tomer Fellow - Metropolitan Policy Program</p>
  <p>Twitter AdieTomer</p>
  <p>But all is not well within our so-called Urban Century. Rapid
  urbanization, rising gentrification, concentrated poverty, and
  shortages of basic infrastructure have combined to create spatial
  inequity in cities and suburbs across the globe. The challenges of
  housing, moving, and employing so many people have led to longer
  travel times, rising housing costs, and unsustainable public spending.
  Moreover, policymakers are questioning traditional policies and
  approaches. The past couple years, we’ve led a project at
  Brookings—Moving to Access—that responds to these spatial challenges
  by promoting the idea of connecting people to opportunities as a new
  foundational principle for 21st century urban development. This
  principle of accessibility is meant to be a corollary to the natural
  questions we ask ourselves everyday about the communities where we
  live: Is this the best location to access employment? Are there nearby
  schools and health services? Is there a market in the neighborhood?
  How can I get from here to there? Such choices are valid for those
  with sufficient income. But what about those with more limited
  resources and thus choices in terms of affordable housing and
  affordable transport? ...</p>
  </blockquote>
  <p>For second website I used the following code:</p>
  <pre><code>url = 'https://www.nytimes.com/2018/01/25/opinion/sunday/silicon-valley-immortality.html'
  
  bs = getPage(url)
  title = bs.find('h1').text
  
  body = bs.find('section', {'name': 'articleBody', 'class': 'meteredContent'}).text
  
  utb = Content(url, title, body)
  
  print('Title: {}'.format(utb.title))
  print('URL: {}\n'.format(utb.url))
  print(utb.body)
  </code></pre>
  <p>Output:</p>
  <blockquote>
  <p>Title: The Men Who Want to Live Forever URL:
  <a href="https://www.nytimes.com/2018/01/25/opinion/sunday/silicon-valley-immortality.html" rel="nofollow noreferrer">https://www.nytimes.com/2018/01/25/opinion/sunday/silicon-valley-immortality.html</a></p>
  <p>Would you like to live forever? Some billionaires, already invincible
  in every other way, have decided that they also deserve not to die.
  Today several biotech companies, fueled by Silicon Valley fortunes,
  are devoted to “life extension” — or as some put it, to solving “the
  problem of death.”It’s a cause championed by the tech billionaire
  Peter Thiel, the TED Talk darling Aubrey de Gray, Google’s
  billion-dollar Calico longevity lab and investment by Amazon’s Jeff
  Bezos. The National Academy of Medicine, an independent group,
  recently dedicated funding to “end aging forever.”As the longevity
  entrepreneur Arram Sabeti told The New Yorker: “The proposition that
  we can live forever is obvious. It doesn’t violate the laws of
  physics, so we can achieve it.” Of all the slightly creepy aspects to
  this trend, the strangest is the least noticed: The people publicly
  championing life extension are mainly men.Not all of them, of course.
  In 2009, Elizabeth Blackburn received the Nobel Prize for her work on
  telomeres, protein caps on chromosomes that may be a key to
  understanding aging. Cynthia Kenyon, the vice president for aging
  research at Calico, studied life extension long before it was cool;
  her form ...</p>
  </blockquote>
  <p>As you can see, the result from the second website don't have &quot;spacing&quot; between paragraphs.</p>
  <p>Thank you for any clarifying explanation, why the same code used on 2 websites with basically the same structure outputs differently (difference in spacing between paragraphs)?</p>