title: Scrapy crawler not completeing all loops in parse function
body: <p>I have this code in my crawler</p>
  
  <pre><code>class StackSpider(InitSpider):
      name = 'stack'
      allowed_domains = ['sitepoint.com']
      start_urls = ["http://www.sitepoint.com"]
      start_page = "http://www.sitepoint.com"
      item = StackItem()
  
      def init_request(self):
  
          return Request(url=self.start_page, callback=self.parse)
  
      def parse(self, response):
  
          hxs = HtmlXPathSelector(response)
          sites = hxs.select('//div[@class="headline_area"]')
          items = []
  
  
          ivar = 1
          for site in sites[:5]:
              item = StackItem()
              log.msg(' LOOP' +str(ivar)+ '', level=log.ERROR)
              item['title'] ="yoo ma"
              request =  Request("http://www.sitepoint.com/getting-to-know-css3-selectors-structural-pseudo-classes/",  callback=self.test1)
              request.meta['item'] = item
              ivar = ivar + 1
              yield request
  
  
      def test1(self, response):
          log.msg('  LOOP 2 \n', level=log.ERROR)
          item = response.meta['item']
          item['desc'] = "test4"
          return item
  </code></pre>
  
  <p>I did it as per <a href="http://doc.scrapy.org/en/latest/topics/request-response.html#passing-additional-data-to-callback-functions" rel="nofollow">documentation</a> but it only works on one loop.
  I mean i can only see in log on screen</p>
  
  <pre><code>LOOP1
  LOOP2
  </code></pre>
  
  <p>It should be repeated 3 times</p>
  
  <p>I tried the different combination of return and yield  so</p>
  
  <ol>
  <li><code>return request</code>  and <code>return item</code> gives output   <code>LOOP1 LOOP2</code></li>
  <li><code>yield request</code> and <code>return item</code> gives output  <code>LOOP1 LOOP1 LOOP1 LOOP2</code></li>
  <li><code>yield request</code> and <code>yield item</code> gives output  <code>LOOP1 LOOP1 LOOP1 LOOP2</code></li>
  <li><code>return request</code> and <code>yield item</code> gives output  <code>LOOP1 LOOP2</code></li>
  </ol>
  
  <p>How can i get <code>LOOP 1 LOOP2 LOOP1 LOOP2 AND so on</code></p>