title: Avoid buffering when parsing stdout with Perl
body: <p>I want to parse the output of an external program (some shell command) line by line using Perl. The command runs continuously, so I put it into a thread and use shared variables to communicate with my main routine.</p>
  <p>Up to now my code looks similar to</p>
  <pre><code>#!/usr/bin/perl
  
  use warnings;
  use strict;
  use threads;
  use threads::shared;
  
  my $var :shared; $var=&quot;&quot;;
  
  threads-&gt;create(
      sub {
          # command writes to stdout each ~100ms
          my $cmd = &quot;&lt;long running command&gt; |&quot;;
          open(README, $cmd) or die &quot;Can't run program: $!\n&quot;;
          while(&lt;README&gt;) {
              my $line = $_;
              # extract some information from line
              $var = &lt;some value&gt;;
              print &quot;Debug\n&quot;;
          }
          close(README);
      }
  );
  
  while(1) {
      # evaluate variable each ~second
      print &quot;$var\n&quot;;
      sleep 1;
  }
  </code></pre>
  <p>For some commands this works perfectly fine and the lines are processed just as they come in. Output would be similar to:</p>
  <pre><code>...
  Debug
  Debug
  ...
  &lt;value 1&gt;
  ...
  Debug
  Debug
  ...
  &lt;value 2&gt;
  ...
  </code></pre>
  <p>However, for other commands, this behaves strange and the lines are being processed block wise. So <code>$var</code> doesn't get updated and <code>Debug</code> is not printed either for some time. Then, the suddenly the output is (similar to):</p>
  <pre><code>...
  &lt;value 1&gt;
  &lt;value 1&gt;
  &lt;value 1&gt;
  ...
  Debug
  Debug
  Debug
  ...
  &lt;value 20&gt;
  </code></pre>
  <p>and <code>$var</code> is set to the last/current value. Then this repeats. The parsing is always delayed and done in blocks while <code>$var</code> is not updated in between.</p>
  <p>First of all: Is there any better/propper way to parse the output of an external program (line by line!) besides using the pipe?</p>
  <p>If not, how can I avoid this behaviour?</p>
  <p>I've read, that using <code>autoflush(1);</code> or <code>$|=1;</code> might be a solution but only for the <strong>&quot;currently selected output channel&quot;</strong>. How would I use that in my context?</p>
  <p>Thank you in advance.</p>