title: Memory usage accumulated in while loop
body: <p>My code includes this <code>while</code> loop:</p>
  
  <pre><code>while A.shape[0] &gt; 0:
      idx = A.score.values.argmax()
      one_center = A.coordinate.iloc[idx]
      # peak_centers and peak_scores are python lists
      peak_centers.append(one_center)
      peak_scores.append(A.score.iloc[idx])
      # exclude the coordinates around the selected peak
      A = A.loc[(A.coordinate &lt;= one_center - exclusion) | (A.coordinate &gt;= one_center + exclusion)]
  </code></pre>
  
  <p><code>A</code> is a pandas <code>DataFrame</code> that looks like this:</p>
  
  <pre><code>   score  coordinate
  0  0.158           1
  1  0.167           2
  2  0.175           3
  3  0.183           4
  4  0.190           5
  </code></pre>
  
  <p>I am trying to find the maximum score (a peak) in <code>A</code>, then exclude some coordinates (a couple hundred in this case) around the previously found peak, then find the next peak and so on. </p>
  
  <p><code>A</code> here is a very large pandas <code>DataFrame</code>. Before running this <code>while</code> loop, the ipython session used 20% of machine memory. I thought that running this <code>while</code> loop would only cause memory consumption to drop because I am excluding some data from the <code>DataFrame</code>. However, what I observe is that memory usage keeps increasing and at some point machine memory is exhausted.</p>
  
  <p>Is there anything I missed here? Do I need to explicitly free memory somewhere? </p>
  
  <p>Here is a short script that can replicate the behavior using random data:</p>
  
  <pre><code>import numpy as np
  import pandas as pd
  
  A = pd.DataFrame({'score':np.random.random(132346018), 'coordinate':np.arange(1, 132346019)})
  peak_centers = []
  peak_scores = []
  exclusion = 147
  while A.shape[0] &gt; 0:
      idx = A.score.values.argmax()
      one_center = A.coordinate.iloc[idx]
      # peak_centers and peak_scores are python lists
      peak_centers.append(one_center)
      peak_scores.append(A.score.iloc[idx])
      # exclude the coordinates around the selected peak
      A = A.loc[(A.coordinate &lt;= one_center - exclusion) | (A.coordinate &gt;= one_center + exclusion)]
  
  # terminated the loop after memory consumption gets to 90% of machine memory
  # but peak_centers and peak_scores are still short lists
  print len(peak_centers)
  # output is 16
  </code></pre>