title: How to use multiprocessing to construct data from a list of files in python
body: <p>I am interested in speeding up my file read times by implementing multiprocessing, but I am having trouble getting data back from each process. The order does matter when all the data is put together and I am using Python 3.9.</p>
  <pre><code># read files from file list in the given indices
  def read_files(files, folder_path):
      raw_data = []
      # loops through all tif files in the given folder and parses the data.
      for file in files:
          if file[-3:] == &quot;tif&quot;:
              curr_frame = Image.open(os.path.join(folder_path, file))
              raw_data.append(np.array(curr_frame))
      return np.asarray(raw_data).astype(np.float64)
  
  
  def run_processes(folder_path=None):
      if folder_path is None:
          global PATH
          folder_path = PATH
      files = os.listdir(folder_path)
  
      start = time.time()
      processes = []
      num_files_per = int(len(files) / os.cpu_count())
      for i in range(os.cpu_count()):
          processes.append(Process(target=read_files, args=(files[(i*num_files_per):((i+1)*num_files_per)], folder_path)))
      for process in processes:
          process.start()
      for process in processes:
          process.join()
      end = time.time()
      print(f&quot;Multi: {end - start}&quot;)
  </code></pre>
  <p>Any help is much appreciated!</p>