title: Python future gets slower after a while
body: <p>I am submitting jobs using python 3 <code>concurrent.futures</code>.</p>
  <pre><code>lock = threading.Lock()
  def do_the_work(item):
      t1, t2 = find_delta(item[1]['file1'], item[1]['file2'])
      datapoint = (t1, t2)
  
      while lock.locked():
         continue
  
      lock.acquire()
      writer.writerow(datapoint)
      lock.release()
      print('Done writing to csv')
  
  with concurrent.futures.ProcessPoolExecutor(max_workers=os.cpu_count()) as executor:
      futures = [executor.submit(do_the_work, item) for item in work_list.items()]
  </code></pre>
  <p>Initially, this code runs very fast. But after a while, i.e. after processing ~50K records it becomes dead-slow.</p>
  <p>Here, I don't see any issues with deadlock. Futures once they can hold the lock, just writing to the file. Then why it's acting like this?</p>
  <p>CPU load becomes very high which is strange as well. While running machine has enough free memory.</p>
  <p>I know I can write the code again with the producer, consumer with a queue. But I don't understand what's wrong with the above code.</p>
  <p>I am wondering is there a way to solve this problem using the <code>concurrent.futures</code>?</p>