title: Is it possible to speed-up python IO?
body: <p>Consider this python program:</p>
  
  <pre><code>import sys
  
  lc = 0
  for line in open(sys.argv[1]):
      lc = lc + 1
  
  print lc, sys.argv[1]
  </code></pre>
  
  <p>Running it on my 6GB text file, it completes in ~ 2minutes.</p>
  
  <p>Question: <strong>is it possible to go faster?</strong></p>
  
  <p>Note that the same time is required by:</p>
  
  <pre><code>wc -l myfile.txt
  </code></pre>
  
  <p>so, I suspect the anwer to my quesion is just a plain "no".</p>
  
  <p>Note also that my real program is doing something more interesting than just counting the lines, so please give a generic answer, <strong>not</strong> line-counting-tricks (like keeping a line count metadata in the file)</p>
  
  <p>PS: I tagged "linux" this question, because I'm interested only in linux-specific answers. Feel free to give OS-agnostic, or even other-OS answers, if you have them.</p>
  
  <p>See also the <a href="https://stackoverflow.com/questions/860893/is-python-automagically-parallelizing-io-and-cpu-or-memory-bound-sections">follow-up question</a></p>