title: Problems with python subprocess and stdout
body: <p>Trying to make a python script that runs some bash jobs (for flashing) in separate threads using <code>subprocess.Popen</code>. While running I want to read the stdout of the processes so I can show some progress to the user in a simple curses interface. So each thread runs something like this:</p>
  
  <pre><code>def worker(panel):
      panel['status'] = 'STARTING'
      panel['process'] = Popen([SCRIPTFILE, panel['ip'], FWFILE, JADFILE, EXEFILE], stdout=PIPE, bufsize=1048576)
      panel['status'] = 'RUNNING'
  
      line = " "
      while line:
          line = panel['process'].stdout.readline()
          panel['status'] = line
      panel['status'] = 'DONE'
  </code></pre>
  
  <p>I see in my UI that the status gets stuck on <code>STARTING</code> in this case, but looking at IP traffic it seems that it actually runs but never returns any output. So it seems as <code>Popen</code> actually runs the job but doesn't return. And as soon as the other processes finish it seems it stops hanging and I get all the data at once as it was buffered. I have seen warnings about reading from stdout on a subprocess, but those warnings seem to be related to other scenarios. And I can't really use <code>communicate()</code> instead as it waits for the process to finish. Any idea what could make <code>Popen</code> hang like this?</p>
  
  <p>Tried to make a minimal version showing the problem here (running with python 2.7 on Linux (change find path in script if windows)):</p>
  
  <pre><code>import curses
  import time
  from subprocess import *
  import threading
  
  def check_done(panels):
      done = True
  
      for panel in panels:
          if panel['status'] != 'DONE':
              done = False
              break
  
      return done
  
  def flash_worker(panel):
      panel['status'] = 'STARTING'
  
      panel['process'] = Popen(["find", "/usr"], stdout=PIPE, bufsize=1048576)
  
      panel['status'] = 'RUNNING'
  
      line = " "
  
      while line:
          line = panel['process'].stdout.readline()
  
          panel['status'] = line
  
      panel['status'] = 'DONE'
  
  def ui_worker(panels):
      stdscr = curses.initscr()
      curses.noecho()
      curses.cbreak()
      stdscr.keypad(1)
      stdscr.scrollok(1)
  
      stdscr.erase()
      stdscr.refresh()
  
      while not check_done(panels):
          y = 0
  
          stdscr.clear()
  
          stdscr.addstr(y, 0, "IP")
          stdscr.addstr(y, 15, "Status")
  
          height, width = stdscr.getmaxyx()
  
          for i in range(0, len(panels)):
              y = y + 1
              stdscr.addstr(y, 0, "%s" % (panels[i]['ip']))
              stdscr.addstr(y, 15, "%s" % (panels[i]['status']))
  
          stdscr.refresh()
          time.sleep(0.1)
  
      curses.nocbreak();
      stdscr.keypad(0);
      curses.echo()
      curses.endwin()
  
  def main():
      panels = []
  
      for i in range(0,25):
          panels.append({ 'ip' : '192.168.0.%d' % (i), 'mac' : '', 'status' : '', 'thread' : None, 'success' : False })
  
      for panel in panels:
          panel['thread'] = threading.Thread(name='Panel %s' % (panel['ip']), target = flash_worker, args = (panel,))
          panel['thread'].start()
  
      try:
          ui_worker(panels)
      except KeyboardInterrupt:
          print "Caught KeyboardInterrupt, terminating..."
  
      for panel in panels:
          panel['thread'].join()
  
  if __name__ == "__main__":
      main()
  </code></pre>