title: Python3 can't pickle _thread.RLock objects on list with multiprocessing
body: <p>I'm trying to parse the websites that contain car's properties(154 kinds of properties). I have a huge list(<em>name is</em> <strong>liste_test</strong>) that consist of 280.000 used car announcement URL.</p>
  
  <pre><code>def araba_cekici(liste_test,headers,engine):
      for link in liste_test:
          try:
              page = requests.get(link, headers=headers)
          .....
          .....
  </code></pre>
  
  <p>When I start my code like that:</p>
  
  <pre><code>araba_cekici(liste_test,headers,engine)
  </code></pre>
  
  <p>It works and getting results. But approximately in 1 hour, I could only obtain 1500 URL's properties. It is very slow, and I must use <strong>multiprocessing</strong>.</p>
  
  <p>I found a result on <a href="https://www.kdnuggets.com/2018/02/web-scraping-tutorial-python.html" rel="noreferrer">here</a> with multiprocessing. Then I applied to my code, but unfortunately, it is not working.</p>
  
  <pre><code>import numpy as np
  import multiprocessing as multi
  
  def chunks(n, page_list):
      """Splits the list into n chunks"""
      return np.array_split(page_list,n)
  
  cpus = multi.cpu_count()
  
  workers = []   
  page_bins = chunks(cpus, liste_test)
  
  
  for cpu in range(cpus):
      sys.stdout.write("CPU " + str(cpu) + "\n")
      # Process that will send corresponding list of pages 
      # to the function perform_extraction
      worker = multi.Process(name=str(cpu), 
                             target=araba_cekici, 
                             args=(page_bins[cpu],headers,engine))
      worker.start()
      workers.append(worker)
  
  for worker in workers:
      worker.join()
  </code></pre>
  
  <p>And it gives:</p>
  
  <pre><code>TypeError: can't pickle _thread.RLock objects
  </code></pre>
  
  <p>I found some kind of responses with respects to this error. But none of them works(at least I can't apply to my code).
  Also, I tried python multiprocess <a href="https://docs.python.org/2/library/multiprocessing.html" rel="noreferrer">Pool</a> but unfortunately it stucks on <strong>jupyter notebook</strong> and seems this code works infinitely.</p>