title: beautifulsoup with an invalid html document
body: <p>I am trying to parse the document <a href="http://www.consilium.europa.eu/uedocs/cms_data/docs/pressdata/en/ecofin/5923en8.htm" rel="nofollow noreferrer">http://www.consilium.europa.eu/uedocs/cms_data/docs/pressdata/en/ecofin/5923en8.htm</a>.
  I want to extract everything before <code>Commission:</code>.
  <img src="https://i.stack.imgur.com/GhhkH.png" alt="enter image description here"></p>
  
  <p>(<strong>I need Beautifulsoup because the second step is to extract countries and person names</strong>)</p>
  
  <p>If i do:</p>
  
  <pre><code>import urllib
  import re
  from bs4 import BeautifulSoup
  url="http://www.consilium.europa.eu/uedocs/cms_data/docs/pressdata/en/ecofin/5923en8.htm"
  soup=BeautifulSoup(urllib.urlopen(url))
  print soup.find_all(text=re.compile("Commission"))
  </code></pre>
  
  <p>The only result I get is :</p>
  
  <pre><code>[u'The Governments of the Member States and the European Commission were represented as follows:']
  </code></pre>
  
  <p>This is the first occurrence of the word, but not the line I am looking for. I think it's because the document is not valid, but not sure. If I look at the source code :</p>
  
  <pre><code>&lt;B&gt;&lt;U&gt;&lt;P&gt;Commission&lt;/B&gt;&lt;/U&gt;:&lt;/P&gt;
  </code></pre>
  
  <p>But if I do a print of <code>soup</code>, I can see the text, with tags reordered:</p>
  
  <pre><code>&lt;u&gt;&lt;b&gt;Commission&lt;/b&gt;&lt;/u&gt;
  </code></pre>
  
  <p>How can I get this element <code>"Commission:"</code>?</p>
  
  <p><em>I am using python 2.7 and Beautifoulsoup 4.3.2.</em></p>
  
  <hr>
  
  <h2>EDIT: SOLVED!</h2>
  
  <p>As suggested alecxe, I replaced the line:</p>
  
  <pre><code>soup=BeautifulSoup(urllib.urlopen(url))
  </code></pre>
  
  <p>with</p>
  
  <pre><code>BeautifulSoup(urllib.urlopen(url), 'html.parser')
  </code></pre>
  
  <p>It works now :).
  Thanks to everyone.</p>
  
  <hr>
  
  <h2>EDIT: similar problems</h2>
  
  <p>I have founs similar problems with the same solution:</p>
  
  <p><a href="https://stackoverflow.com/questions/17698836/beautiful-soup-4-find-all-dont-find-links-that-beautiful-soup-3-finds">Beautiful Soup 4 find_all don&#39;t find links that Beautiful Soup 3 finds</a></p>
  
  <p><a href="https://stackoverflow.com/questions/16322862/beautiful-soup-findall-doent-find-them-all">Beautiful Soup findAll doen&#39;t find them all</a></p>