title: Close a scrapy spider when a condition is met and return the output object
body: <p>I have made a spider to get reviews from a page like this <a href="http://www.flipkart.com/moto-x-play/product-reviews/ITMEAJTQYHRNKWRZ?pid=MOBEAJTQGSBHEFHA&amp;type=all&amp;sort=most_recent" rel="nofollow">here</a> using scrapy. I want product reviews only till a certain date(2nd July 2016 in this case). I want to close my spider as soon as the review date goes earlier than the given date and return the items list.
  Spider is working well but my problem is that i am not able to close my spider if the condition is met..if i raise an exception, spider closes without returning anything.
  Please suggest the best way to close the spider manually. Here is my code:</p>
  
  <pre><code>from scrapy.spiders import CrawlSpider, Rule
  from scrapy.linkextractors import LinkExtractor
  from scrapy import Selector
  from tars.items import FlipkartProductReviewsItem
  import re as r
  import unicodedata
  from datetime import datetime 
  
  class Freviewspider(CrawlSpider):
      name = "frs"
      allowed_domains = ["flipkart.com"]
      def __init__(self, *args, **kwargs):
          super(Freviewspider, self).__init__(*args, **kwargs)
          self.start_urls = [kwargs.get('start_url')]
  
  
      rules = (
          Rule(LinkExtractor(allow=(), restrict_xpaths=('//a[@class="nav_bar_next_prev"]')), callback="parse_start_url", follow= True),
  )
  
  
      def parse_start_url(self, response):
  
          hxs = Selector(response)
          titles = hxs.xpath('//div[@class="fclear fk-review fk-position-relative line "]')
  
          items = []
  
          for i in titles:
  
              item = FlipkartProductReviewsItem()
  
              #x-paths:
  
              title_xpath = "div[2]/div[1]/strong/text()"
              review_xpath = "div[2]/p/span/text()"
              date_xpath = "div[1]/div[3]/text()"
  
  
  
              #field-values-extraction:
  
              item["date"] = (''.join(i.xpath(date_xpath).extract())).replace('\n ', '')
              item["title"] = (''.join(i.xpath(title_xpath).extract())).replace('\n ', '')
  
              review_list = i.xpath(review_xpath).extract()
              temp_list = []
              for element in review_list:
                  temp_list.append(element.replace('\n ', '').replace('\n', ''))
  
              item["review"] = ' '.join(temp_list)
  
              xxx = datetime.strptime(item["date"], '%d %b %Y ')
              comp_date = datetime.strptime('02 Jul 2016 ', '%d %b %Y ')
              if xxx&gt;comp_date:
                  items.append(item)
              else:
                  break
  
          return(items)
  </code></pre>