title: How would you use scrapy to select an option from a select tag on a site?
body: <p>I am learning how to use Scrapy. I was following the tutorial for simulating user login with scrapy and then collecting data after successfully being logged in (code below). However, how would one program scrapy to select a certain option from a drop down list if it was part of the login process?</p>
  
  <pre><code>class LoginSpider(BaseSpider):
      name = 'example.com'
      start_urls = ['http://www.example.com/users/login.php']
  
      def parse(self, response):
          return [FormRequest.from_response(response,
                      formdata={'username': 'john', 'password': 'secret'},
                      callback=self.after_login)]
  
      def after_login(self, response):
          # check login succeed before going on
          if "authentication failed" in response.body:
              self.log("Login failed", level=log.ERROR)
              return
  
          # continue scraping with authenticated session...
  </code></pre>
  
  <p>Select tag:</p>
  
  <pre><code>&lt;select&gt;
  
      &lt;option&gt;Value 1&lt;/option&gt;
      &lt;option&gt;Value 2&lt;/option&gt;
      &lt;option&gt;Value 3&lt;/option&gt;
  
  &lt;/select&gt;
  </code></pre>
  
  <p>How would I program scrapy to select, say Value 1, in a drop down list, in addition to entering the login information.</p>