title: memory leak when using multiprocessing
body: <p>As in the title, I'm struggling with memory leak when using <code>multiprocessing</code>. I know the question like this has been asked before, but I still cannot find the right solution for my problem.</p>
  
  <p>I have a list of RGB images (<code>30.000</code> total). I need to read each image, process all three RGB channels, then keep the result in the memory (<em>to be saved in <code>1</code> big file later</em>)</p>
  
  <p>I'm trying to use something like this:</p>
  
  <pre><code>import multiprocessing as mp
  import random
  import numpy as np
  
  
  # Define an output queue to store result
  output = mp.Queue()
  
  # define a example function
  def read_and_process_image(id, output):
      result = np.random.randint(256, size=(100, 100, 3)) #fake an image
      output.put(result)
  
  # Setup a list of processes that we want to run
  processes = [mp.Process(target=read_and_process_image, args=(id, output)) for id in range(30000)]
  
  # Run processes
  for p in processes:
      p.start()
  
  # # Exit the completed processes
  # for p in processes:
  #     p.join()
  
  # Get process results from the output queue
  results = [output.get() for p in processes]
  
  print(results)
  </code></pre>
  
  <p>This code uses a lot of memory. <a href="https://stackoverflow.com/questions/10721915/shared-memory-objects-in-python-multiprocessing">This answer</a> explained the problem, but I cannot find the way to apply it to my code. Any suggestion? Thanks!</p>
  
  <p><strong>Edit</strong>: I also try <code>joblib</code> and the <code>Pool</code> class, but the code won't use all the cores like I expected (I see no difference between using normal <code>for</code> loop with these 2 cases)</p>