title: Running scrapy spider and progress bar simuntaneously
body: <p>I am trying to run a progress bar in my python GUI while a scrapy spider does its job. I have tried several multithreading solutions, like the one below, but none of them have worked for me.</p>
  
  <pre><code>p1 = Process(target=RunSpiders())
  p1.start()
  p2 = Process(target=ProgressBar())
  p2.start()
  c = p1.join()
  p2.join()
  
  if c != None:
      p2.stop()
  </code></pre>
  
  <p>The <code>RunSpiders()</code> method runs two spiders, not using them directly, but creating the cmd command and executing it. The program seems to be executing only the process placed first at each time. At first I thought it was due to the fact that scrapy is an asynchronous framework, but I have more code than the one related to the cmd command execution in the <code>RunSpider()</code> method (such as prints) and those do not appear either, so I think my mistake may be caused by a bad use of <code>multiprocessing</code>.</p>
  
  <p>Code for <code>ProgressBar()</code>:</p>
  
  <pre><code>def ProgressBar():
      progress = ttk.Progressbar(window, orient=tk.HORIZONTAL, length=int(window_width/2), mode='indeterminate')
      progress.place(x=int(window_width/4), y=y)
      i = 0
      while True:
          if i == 100:
              i = 0
          progress['value'] = i
          window.update_idletasks()
          i = i + 0.01
      print("···")
  </code></pre>
  
  <p>Code for <code>Runspiders():</code></p>
  
  <pre><code>def RunSpiders():
      codeS1 = 0
      codeS2 = 0
          if s1.get():
              os.chdir('../')
              print(os.getcwd())
              cmd = "scrapy crawl Spider1"
              codeS1 = os.system(cmd)
          if s2.get():
              os.chdir('../')
              print(os.getcwd())
              cmd = "scrapy crawl Spider2"
              codeS2 = os.system(cmd)
      codeS = codeS1 + codeS2
      return codeS
  </code></pre>