title: Removing tags from a BeautifulSoup object
body: <p>I am having some difficulty scraping a table from a forum website (with 50 lines per page). My code, as written, works for 95% of the content but adds a table break when a bold or italicized line is present and I do not get the full results</p>
  
  <p>The majority of the pages have the tags like this:</p>
  
  <pre><code>&lt;table&gt;
    &lt;td&gt;
      content_1
    &lt;/td&gt;&lt;td&gt;
      content_2
    &lt;/td&gt;&lt;td&gt;
      content_3
    &lt;/td&gt;
    ...
  &lt;/table&gt;
  </code></pre>
  
  <p>Though when a bold or italicized phrase is present, for whatever reason a random  tag is added and I only end up with the number of elements up until the tag.</p>
  
  <pre><code>&lt;table&gt;
    &lt;td&gt;
      content_1
    &lt;/td&gt;&lt;td&gt;
        &lt;b&gt;&lt;i&gt;content_2&lt;/b&gt;&lt;/i&gt; *&lt;/table&gt;*
    &lt;/td&gt;&lt;td&gt;
      content_3
    &lt;/td&gt;
  ...
  &lt;/table&gt;
  </code></pre>
  
  <p>I am not sure why this tag shows up (it does not appear when I inspect the element on the webpage).</p>
  
  <p>Does anyone have any suggestions on how to deal with this? I have a few ideas but none of them have worked out yet.</p>
  
  <ul>
  <li><p>Convert the beautifulsoup object to string then remove the tag and convert back to beautifulsoup - has not quite worked yet, but seems a little excessive and will slow down the program...</p></li>
  <li><p>Just skip soup and use pd.read_html - haven't gotten it to work yet.</p></li>
  </ul>
  
  <p>Or if anyone has suggestions to an alternative approach, that would be appreciated as well. Thanks!</p>
  
  <p>edit - figured it would help if I just added my code. If main block has three examples. First provides no results, second has partial results, and the third is full results.</p>
  
  <pre><code>def get_page(URL):
      '''
      INPUT: url
      OUTPUT: pandas dataframe with message board info
      '''
      content = requests.get(URL).content
      soup = BeautifulSoup(content, "html.parser")
      rows = list(soup.find('table', id="ctl00_CP1_gv"))
      table_lst = []
      for row in rows[2:-2]:
        cell_lst = [cell for cell in list(row)[1:5]]
        table_lst.append(cell_lst)
      return pd.DataFrame(table_lst)
  
  if __name__ == '__main__':
      url1 = "https://investorshub.advfn.com/Cal-Bay-International-Inc-CBYI-5520/?NextStart=35092"
      url2 = "https://investorshub.advfn.com/Cal-Bay-International-Inc-CBYI-5520/?NextStart=35099"
      url3 = "https://investorshub.advfn.com/Cal-Bay-International-Inc-CBYI-5520/?NextStart=1000"
      df1 = get_page(url1)
      df2 = get_page(url2)
      df3 = get_page(url3)
  </code></pre>