title: Screen Scraping in Python
body: <p>I'm new to the whole concept of screen scraping in Python, although I've done a bit of screen scraping in R. I'm trying to scrape the Yelp website. I'm trying to scrape the names of each insurance agency which the yelp search returns. With most scraping tasks, I'm able to perform the following task, but always have a hard time going forward with parsing the xml. </p>
  
  <pre><code>import urllib2
  from BeautifulSoup import BeautifulSoup
  
  soup = BeautifulSoup(urllib2.urlopen('http://www.yelp.com/search?find_desc=insurance+agency&amp;ns=1&amp;find_loc=Austin').read())
  
  print soup
  </code></pre>
  
  <p>So when scraping a site, what are the steps that one should follow? Is there a set of necessary actions that one needs to take each time they attempt to scrape a site?</p>
  
  <p>I'm running Python 2.6 on Ubuntu 10.10 </p>
  
  <p>I realize that this may be a poor SO question as outlined in the faq, but I'm hoping someone can provide some general guidelines and things to consider when scraping a site.</p>