title: Windows disk usage issues with python
body: <p>I am executing the python code that follows.</p>
  
  <p>I am running it on a folder ("articles") which has a couple hundred subfolders and 240,226 files in all.</p>
  
  <p>I am timing the execution. At first the times were pretty stable but went non-linear after 100,000 files. Now the times (I am timing at 10,000 file intervals) can go non_linear after 30,000 or so (or not). </p>
  
  <p>I have the Task Manager open and correlate the slow-downs to 99% Disk usage by python.exe. I have done gc-collect(). dels etc., turned off Windows indexing. I have re-started Windows, emptied the trash (I have a few hundred GBs free). Nothing helps, the disk usage seems to be getting more erratic if anything.</p>
  
  <p>Sorry for the long post - Thanks for the help</p>
  
  <pre><code>def get_filenames():
      for (dirpath, dirnames, filenames) in os.walk("articles/"):
          dirs.extend(dirnames)
  
      for dir in dirs:
          path = "articles" + "\\" + dir        
          nxml_files.extend(glob.glob(path + "/*.nxml"))
  
      return nxml_files
  
  def extract_text_from_files(nxml_files):  
      for nxml_file in nxml_files:       
          fast_parse(nxml_file)
  
  def fast_parse(infile):
      file = open(infile,"r")
      filetext = file.read()
      tag_breaks = filetext.split('&gt;&lt;')
      paragraphs = [tag_break.strip('p&gt;').strip('&lt;/') for tag_break in tag_breaks if tag_break.startswith('p&gt;')]
  
  def run_files(): 
      nxml_files = get_filenames()
      extract_text_from_files(nxml_files)
  
  if __name__ == "__main__":    
      run_files()
  </code></pre>