title: Reading stderr of subprocess while it is executing
body: <p>I'd like to read what is written to stderr by a subprocess while it is executing.</p>
  
  <p>However, when I use this script that I've written, stderr does not seem to have anything for me to read until the subprocess has exited.</p>
  
  <pre><code>#!/usr/bin/env python2
  
  import sys
  from subprocess import Popen, PIPE, STDOUT
  
  if len(sys.argv) &lt; 2:
      print "Please provide a command"
      sys.exit(1)
  
  sub = Popen(sys.argv[1:], stdout=PIPE, stderr=STDOUT)
  
  for i, line in enumerate(sub.stdout):
      sys.stdout.write("%d: %s" % (i, line))
  </code></pre>
  
  <p><strong>edit:</strong></p>
  
  <p>Ok, I've gotten closer now. If I specify the number of bytes to read it overcomes the buffering.</p>
  
  <pre><code>#!/usr/bin/env python2
  
  import sys 
  from subprocess import Popen, PIPE, STDOUT
  
  if len(sys.argv) &lt; 2:
      print "Please provide a command"
      sys.exit(1)
  
  sub = Popen(sys.argv[1:], stdout=PIPE, stderr=STDOUT)
  
  i = 0 
  while sub.poll() is None:
      line = sub.stdout.read(64)
      line.strip("\b")
      sys.stdout.write("%d: %s\n" % (i, line))
      i += 1
  </code></pre>
  
  <p>Snippet of output:</p>
  
  <pre class="lang-none prettyprint-override"><code>58:  86 q=21.0 size= 4541841kB time=00:00:22.08 bitrate=1685014.2kbi
  frame=  567 fps= 86 q=22.0 size= 4543667kB time=00:00:2
  frame=  621 fps= 87 q=20.0 sizs/s    
  frame= 4545352kB time=00:00:26.11 bitrate=1425939.2kbits/s    
  62: =  686 fps= 90 q=12.0 size= 4546970kB time=00:00:28.89 bitrate=1
  frame=  758 fps= 93 q=25.0 size= 4548534kB t
  frame=  794 fps= 92 bitrate=1168185.5kbits/s    
  65:  q=27.0 size= 4550901kB time=00:00:33.40 bitrate=1115897.0kbits/
  frame=  827 fps= 91 q=27.0 size= 4552324kB time=00:00:34.7
  frame=  857 fps= 89 q=26.0 size=  
  frame=  254kB time=00:00:36.12 bitrate=1032874.9kbits/s    
  69: 892 fps= 88 q=25.0 size= 4556598kB time=00:00:37.36 bitrate=9988
  frame=  948 fps= 89 q=19.0 size= 4558565kB time=
  frame= 1006 fps= 90 q=19937320.4kbits/s    
  72: .0 size= 4560139kB time=00:00:42.16 bitrate=885880.0kbits/s    
  73: frame= 1060 fps= 91 q=19.0 size= 4561958kB time=00:00:44.49 bitr
  frame= 1122 fps= 93 q=18.0 size= 4563460
  frame= 1173 fps=0:47.08 bitrate=793898.4kbits/s
  </code></pre>
  
  <p>It looks like my problem now is that ffmpeg is using backspace characters or similar to mess with stdout. Not sure what's happening here.</p>