title: using beautifulsoup decompose() to get rid of multiple unwanted tags
body: <p>I'm trying to clean up HTML so that all I have is the relevant text I want. The following code cleans up the superscript tags with the first defined function, I wan't to do the same thing but with the 'h4', 'h1', 'a', and 'li' tags before I use .get_text.</p>
  
  <pre><code>import requests
  from bs4 import BeautifulSoup
  url = "https://www.biblegateway.com/passage/?search=Luke+14%3A12-14&amp;version=NIV"
  page = requests.get(url)
  soup = BeautifulSoup(page.content, "html.parser")
  
  def supClean(verseWithSup):
      for sup in verseWithSup:
          verseWithSup.sup.decompose()
      return verseWithSup
  
  def verseExtract(soup):
      verseName = soup.find(class_="passage-display-bcv").get_text()
      verseWithSup = soup.find(class_="passage-text")
      verseBody = supClean(verseWithSup).get_text()
      return verseName, verseBody
  
  verseName, verseBody = (verseExtract(soup))
  
  print(verseName)
  print(verseBody)
  </code></pre>
  
  <p>I get this right now:</p>
  
  <blockquote>
    <p>Luke 14:12-14New International Version (NIV) Then Jesus said to his
    host, “When you give a luncheon or dinner, do not invite your friends,
    your brothers or sisters, your relatives, or your rich neighbors; if
    you do, they may invite you back and so you will be repaid. But when
    you give a banquet, invite the poor, the crippled, the lame, the
    blind, 14 and you will be blessed. Although they cannot repay you, you
    will be repaid at the resurrection of the righteous.”  Cross
    references:Luke 14:13 : ver 21 Luke 14:14 : Ac 24:15</p>
    
    <p>New International Version (NIV) Holy Bible, New International
    Version®, NIV® Copyright ©1973, 1978, 1984, 2011 by Biblica, Inc.®
    Used by permission. All rights reserved worldwide.</p>
  </blockquote>
  
  <p>But all I wan't is this:</p>
  
  <blockquote>
    <p>Then Jesus said to his host, “When you give a luncheon or dinner, do
    not invite your friends, your brothers or sisters, your relatives, or
    your rich neighbors; if you do, they may invite you back and so you
    will be repaid. But when you give a banquet, invite the poor, the
    crippled, the lame, the blind, 14 and you will be blessed. Although
    they cannot repay you, you will be repaid at the resurrection of the
    righteous.”</p>
  </blockquote>
  
  <p>This is what the HTML looks like:</p>
  
  <pre><code>&lt;div class="passage-text"&gt;
   &lt;div class="passage-wrap"&gt;
    &lt;div class="passage-content passage-class-0"&gt;
     &lt;div class="version-NIV result-text-style-normal text-html "&gt;
      &lt;h1 class="passage-display"&gt;
       &lt;span class="passage-display-bcv"&gt;
        Luke 14:12-14
       &lt;/span&gt;
       &lt;span class="passage-display-version"&gt;
        New International Version (NIV)
       &lt;/span&gt;
      &lt;/h1&gt;
      &lt;p&gt;
       &lt;span class="text Luke-14-12" id="en-NIV-25566"&gt;
        Then Jesus said to his host,
        &lt;span class="woj"&gt;
         “When you give a luncheon or dinner, do not invite your friends, your brothers or sisters, your relatives, or your rich neighbors; if you do, they may invite you back and so you will be repaid.
        &lt;/span&gt;
       &lt;/span&gt;
       &lt;span class="text Luke-14-13" id="en-NIV-25567"&gt;
        &lt;span class="woj"&gt;
         But when you give a banquet, invite the poor, the crippled, the lame, the blind,
         &lt;sup class="crossreference" data-cr="#cen-NIV-25567A" data-link='(&amp;lt;a href="#cen-NIV-25567A" title="See cross-reference A"&amp;gt;A&amp;lt;/a&amp;gt;)'&gt;
         &lt;/sup&gt;
        &lt;/span&gt;
       &lt;/span&gt;
       &lt;span class="text Luke-14-14" id="en-NIV-25568"&gt;
        &lt;span class="woj"&gt;
         &lt;sup class="versenum"&gt;
          14
         &lt;/sup&gt;
         and you will be blessed. Although they cannot repay you, you will be repaid at the resurrection of the righteous.”
         &lt;sup class="crossreference" data-cr="#cen-NIV-25568B" data-link='(&amp;lt;a href="#cen-NIV-25568B" title="See cross-reference B"&amp;gt;B&amp;lt;/a&amp;gt;)'&gt;
         &lt;/sup&gt;
        &lt;/span&gt;
       &lt;/span&gt;
      &lt;/p&gt;
      &lt;div class="crossrefs hidden"&gt;
       &lt;h4&gt;
        Cross references:
       &lt;/h4&gt;
       &lt;ol&gt;
        &lt;li id="cen-NIV-25567A"&gt;
         &lt;a href="#en-NIV-25567" title="Go to Luke 14:13"&gt;
          Luke 14:13
         &lt;/a&gt;
         :
         &lt;a class="crossref-link" data-bibleref="Luke 14:21" href="/passage/?search=Luke+14%3A21&amp;amp;version=NIV"&gt;
          ver 21
         &lt;/a&gt;
        &lt;/li&gt;
        &lt;li id="cen-NIV-25568B"&gt;
         &lt;a href="#en-NIV-25568" title="Go to Luke 14:14"&gt;
          Luke 14:14
         &lt;/a&gt;
         :
         &lt;a class="crossref-link" data-bibleref="Acts 24:15" href="/passage/?search=Acts+24%3A15&amp;amp;version=NIV"&gt;
          Ac 24:15
         &lt;/a&gt;
        &lt;/li&gt;
       &lt;/ol&gt;
      &lt;/div&gt;
      &lt;!--end of crossrefs--&gt;
     &lt;/div&gt;
     &lt;div class="publisher-info-bottom with-single"&gt;
      &lt;strong&gt;
       &lt;a href="/versions/New-International-Version-NIV-Bible/"&gt;
        New International Version
       &lt;/a&gt;
       (NIV)
      &lt;/strong&gt;
      &lt;p&gt;
       Holy Bible, New International Version®, NIV® Copyright ©1973, 1978, 1984, 2011 by
       &lt;a href="http://www.biblica.com/"&gt;
        Biblica, Inc.®
       &lt;/a&gt;
       Used by permission. All rights reserved worldwide.
      &lt;/p&gt;
     &lt;/div&gt;
    &lt;/div&gt;
   &lt;/div&gt;
  &lt;/div&gt;
  </code></pre>