title: How can I print and display subprocess stdout and stderr output without distortion?
body: <p>Maybe there's someone out in the ether that can help me with this one. (I have seen a number of similar questions to this on SO, but none deal with both standard out and standard error or deal with a situation quite like mine, hence this new question.)</p>
  
  <p>I have a python function that opens a subprocess, waits for it to complete, then outputs the return code, as well as the contents of the standard out and standard error pipes. While the process is running, I'd like to also display the output of both pipes as they are populated. My first attempt has resulted in something like this:</p>
  
  <pre><code>process = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  
  stdout = str()
  stderr = str()
  returnCode = None
  while True:
      # collect return code and pipe info
      stdoutPiece = process.stdout.read()
      stdout = stdout + stdoutPiece
      stderrPiece = process.stderr.read()
      stderr = stderr + stderrPiece
      returnCode = process.poll()
  
      # check for the end of pipes and return code
      if stdoutPiece == '' and stderrPiece == '' and returnCode != None:
          return returnCode, stdout, stderr
  
      if stdoutPiece != '': print(stdoutPiece)
      if stderrPiece != '': print(stderrPiece)
  </code></pre>
  
  <p>There's a couple problems with this though. Because <code>read()</code> reads until an <code>EOF</code>, the first line of the <code>while</code> loop will not return until the subprocess closes the pipe.</p>
  
  <p>I could replace the <code>read()</code> in favor of <code>read(int)</code> but the printed output is distorted, cut off at the end of the read characters. I could <code>readline()</code> as a replacement, but the printed output is distorted with alternating lines of output and errors when there are many of both that occur at the same time.</p>
  
  <p>Perhaps there's a <code>read-until-end-of-buffer()</code> variant that I'm not aware of? Or maybe it can be implemented?</p>
  
  <p>Maybe it's best to implement a <code>sys.stdout</code> wrapper as suggested in this <a href="https://stackoverflow.com/questions/107705/python-output-buffering/107717#107717">answer to another post</a>? I would only want to use the wrapper in this function, however.</p>
  
  <p>Any other ideas from the community?</p>
  
  <p>I appreciate the help! :)</p>
  
  <p><em>EDIT</em>: The solution really should be cross-platform, but if you have ideas that aren't, please share them away to keep the brainstorming going.</p>
  
  <hr>
  
  <p>For another one of my python subprocess head scratchers, take a look at another of my questions on <a href="https://stackoverflow.com/q/7560708/608905">accounting for subprocess overhead in timing</a>.</p>