title: How to make rules of CrawlSpider context-sensitive?
body: <p>I notice that, the <code>rule</code> of <code>CrawlSpider</code> extract urls on every none-leaf pages.<br>
  Can I enable <code>rule</code> only when current page meets some condition (for example: url matches a regex)?</p>
  
  <p>I have two pages:</p>
  
  <hr>
  
  <pre><code>-------------------Page A-------------------
  Page URL: http://www.site.com/pattern-match.html
  --------------------------------------------
  
  - [link](http://should-extract-this)
  - [link](http://should-extract-this)
  - [link](http://should-extract-this)
  
  --------------------------------------------
  </code></pre>
  
  <hr>
  
  <pre><code>--------------------Page B--------------------
  Page URL: http://www.site.com/pattern-not-match.html
  -----------------------------------------------
  
  - [link](http://should-not-extract-this)
  - [link](http://should-not-extract-this)
  - [link](http://should-not-extract-this)
  
  -----------------------------------------------
  </code></pre>
  
  <hr>
  
  <p>So, the rule should only extract urls from PageA. How to do it? Thanks!</p>