title: Scrapy - How to run spiders with AWS Lambda functions?
body: <p>Currently I have two small projects using Scrapy. One project is basically to scrape URL's, while the other is only to scrape products of the scrape URL's. The directory structure is this:</p>
  
  <pre><code>.
  ├── requirements.txt
  ├── .venv
  ├── url
  |   ├── geckodriver
  |   ├── scrapy.cfg
  |   ├── url
  |   |   ├── items.py
  |   |   ├── middlewares.py
  |   |   ├── pipelines.py
  |   |   ├── settings.py
  |   |   ├── spiders
  |   |   |    ├── store1.py
  |   |   |    ├── store2.py
  |   |   |    ├── ...
  ├── product
  |   ├── geckodriver
  |   ├── scrapy.cfg
  |   ├── product
  |   |   ├── items.py
  |   |   ├── middlewares.py
  |   |   ├── ...
  </code></pre>
  
  <p>When I want to run a spider using the command, I always must follow this path: <code>~/search/url$ scrapy crawl store1</code> or <code>~/search/product$ scrapy crawl store1</code>. </p>
  
  <p>How can I deploy and run this project using AWS lambda functions?</p>