title: How to stop data overflowing onto next line when exporting to csv file and how to catch HTTP error using Python
body: <p>I am scraping the title, date, and body text from each article from Politico's archive of articles. Like any article, it includes a lot of text. I want each article to be printed on my csv file in one long line. However, a few of my articles will overflow onto the next line. </p>
  
  <p>Here is a screenshot of my csv where it is doing what it is supposed to be doing: </p>
  
  <p><a href="https://imgur.com/a/xMoSUKi" rel="nofollow noreferrer">https://imgur.com/a/xMoSUKi</a></p>
  
  <p>Here is a screenshot of my csv where it is overflowing onto the next line: </p>
  
  <p><a href="https://imgur.com/a/G6nX73P" rel="nofollow noreferrer">https://imgur.com/a/G6nX73P</a></p>
  
  <p>Additionally, I want to be able to catch an HTTP error for when the website blocks me for trying to access their pages too much (even with random delays this happens), and I'm not sure what to write for this. I've written a while loop for when I'm still within the archive page amount (there are 414 pages of articles in their archive), so it will keep trying my main function even after we get blocked so I can pick up where it left off.</p>
  
  <p>For the newline issue, when scraping the article text, I've replaced each newline with this symbol "~" (I can just go back and delete it once I've finished scraping). I tried replacing each newline with a blank space (" "), but I can see what is going on more with the ~ symbol. However, it does not seem to replace every new line as it overflows onto the next row of the csv. I've also tried increasing the column width in my csv before running the program but this doesn't have an effect.</p>
  
  <pre><code>import time
  from random import randint
  from urllib.request import urlopen
  
  
  from bs4 import BeautifulSoup
  
  HEADERS = ["Title", "Date", "Article"]
  
  
  def count_rows(csv_path: str) -&gt; int:
      with open(csv_path, encoding='utf-8') as f:
          reader = csv.DictReader(f)
          return len(list(reader))
  
  
  def write_articles(csv_path: str, articles: list):
      # note it's in append mode
      # write mode would delete everything and start fresh
      with open(csv_path, 'a', encoding='utf-8', newline='\n') as f:
          writer = csv.DictWriter(f,
                                  quoting=csv.QUOTE_MINIMAL,
                                  fieldnames=HEADERS)
          writer.writerows(articles)
  
  
  def init_csv(csv_path: str):
      with open(csv_path, 'w', encoding='utf-8', newline=None) as f:
          writer = csv.DictWriter(f, fieldnames=HEADERS, quoting=csv.QUOTE_MINIMAL)
          writer.writeheader()
  
  
  def get_page_soup(url: str) -&gt; BeautifulSoup:
      response = urlopen(url)
      html = response.read()
  
      soup = BeautifulSoup(html, "lxml")
      return soup
  
  
  def scrape_article(url: str) -&gt; dict:
      article_soup = get_page_soup(url)
  
      # Limits to div class = story-text tag (where article text is)
      story_el = article_soup.select_one('.story-text')
  
      # find date
      date = story_el.select_one('.timestamp time')['datetime']
  
      # find title
      title = story_el.find('h1').text
  
      # find body text
      article_text = ''
      for p in story_el.find_all('p'):
          article_text += p.text + ' '                           
          article_text = article_text.replace("\r","~")
          article_text = article_text.replace("\n","~")
  
      return {'Title': title, 'Date': date, 'Article': article_text }
  p=0
  def main():
      csvfile = "C:/Users/katew/Dropbox/granularitygrowth/Politico/pol.csv"
  
      try:
          record_count = count_rows(csvfile)
      except FileNotFoundError:
          init_csv(csvfile)
          print('Initialized CSV file')
          record_count = 0
  
      article_per_page = 10
      page = record_count // article_per_page
  
      print('Continuing from page', page)
  
      for p in range(page + 1, 415):
          articles = []
          url = "https://www.politico.com/newsletters/playbook/archive/%d" % p
          soup = get_page_soup(url)
          article_links = soup.select('article.story-frag.format-l')
  
          # Each article link on page
          for article in article_links:
              link = article.select_one('a[target=_top]')['href']
              scraped_article = scrape_article(link)
              #print(scraped_article)
              articles.append(scraped_article)
              #time.sleep(randint(3, 8))
  
          write_articles(csvfile, articles)
          print('Finished page', p)
          #time.sleep(randint(3, 8))
  
  while p &lt; 415 :
      try:
          if __name__ == '__main__':
              main()
      except urllib2.error.HTTPError:
        print('Politico website blocked us. Starting from where we were stopped.')
  </code></pre>
  
  <p>Notice that I've commented out the delays for now since they make it so time consuming to test my code.</p>
  
  <p>I expect my csv file to include each article title, date and body text on one single line. I also expect my code to be able to pick up from where it was blocked using a while loop.</p>