title: Python 3.4 Multiprocess throws TypeError("cannot serialize '_io.BufferedReader' object",)
body: <p>Recently I wrote a multiprocess code in Python 3.4 to download some images, it's working blazingly fast at first, then I get the following error and cannot start the program anymore.</p>
  
  <pre><code>Traceback (most recent call last):
    File "multiprocessing_d.py", line 23, in &lt;module&gt;
      main()
    File "multiprocessing_d.py", line 16, in main
      p.map(download, lines)
    File "/usr/local/lib/python3.4/multiprocessing/pool.py", line 260, in map
      return self._map_async(func, iterable, mapstar, chunksize).get()
    File "/usr/local/lib/python3.4/multiprocessing/pool.py", line 608, in get
      raise self._value
  multiprocessing.pool.MaybeEncodingError: Error sending result: '&lt;multiprocessing.pool.ExceptionWithTraceback object at 0x7f1e047f32e8&gt;'. Reason: 'TypeError("cannot serialize '_io.BufferedReader' object",)'
  </code></pre>
  
  <p>My code is as following<br>
  download_helper.py</p>
  
  <pre><code>        import sys
          import os
          from pathlib import Path
  
          url_prefix = r"Some prefix"
  
          def setup_download_dir(dictionary):
              download_dir = Path(dictionary)
              if not download_dir.exists():
                  download_dir.mkdir()
              return dictionary
  
          def download_link(dictionary, line):
              from urllib.request import urlretrieve
              itemid = line.split()[0].decode()
              link = line.split()[1].decode()
              if (link.startswith("http")):
                  image_url = link
              else:
                  image_url = url_prefix + link
              if os.path.isfile(dictionary + "/" + itemid + ".jpg"):
                  #print("Already have " + itemid + ".jpg")
                  pass
              else:
                  urlretrieve(image_url, dictionary + "/" + itemid + ".jpg")
  </code></pre>
  
  <p>multiprocessing_d.py</p>
  
  <pre><code>        from functools import partial
          from multiprocessing.pool import Pool
          import sys
          from time import time
          from download_helper import setup_download_dir, download_link
  
          def main():
              file_path = sys.argv[1]
              dic_path = sys.argv[2]
              download_dir = setup_download_dir(dic_path)
              download = partial(download_link, download_dir)
              with open(file_path, 'rb') as f:
                  lines = f.readlines()
                  ts = time()
                  p = Pool(processes=16, maxtasksperchild=1)
                  p.map(download, lines)
                  p.close()
                  p.join()
                  print('Took {}s'.format(time() - ts))
                  f.close()
  
          if __name__ == "__main__":
              main()
  </code></pre>
  
  <p>I've tried to search online but didn't find much information useful. My suspect is that there might be some exception raised in urlretrieve, but I don't know how to debug it. Any comments or suggestions would be appreciated!!</p>
  
  <p>James</p>