title: python monitor a log file non blocking
body: <p>I have a test stub that will write several log messages to the system log. </p>
  
  <p>But, this system log gets updated by many other applications as well. So, basically, I want to do a <code>tail -f system.log | grep "application name"</code> to get only the appropriate log messages. </p>
  
  <p>I was looking at dbaez generator tricks, and I am trying to combine the both <a href="http://www.dabeaz.com/generators/follow.py" rel="nofollow">http://www.dabeaz.com/generators/follow.py</a> and <a href="http://www.dabeaz.com/generators/apachelog.py" rel="nofollow">http://www.dabeaz.com/generators/apachelog.py</a></p>
  
  <p>So, in my <code>__main__()</code>, I have something like this:</p>
  
  <pre><code>try:
     dosomeprocessing()     #outputs stuff to the log file
  </code></pre>
  
  <p>And within dosomeprocessing(), I run a loop, and for each loop, I want to see if there are any new log messages caused by my application, and not necessarily print it out, but store them somewhere to do some validation.</p>
  
  <pre><code>    logfile = open("/var/adm/messages","r")
      loglines = follow(logfile)
      logpats = r'I2G(JV)'
      logpat = re.compile(logpats)
      groups = (logpat.match(line) for line in loglines)
      for g in groups:
          if g:
              print g.groups()
  </code></pre>
  
  <p>The log looks something like : </p>
  
  <pre><code>Feb  4 12:55:27 Someprocessname.py I2G(JV)-300[20448]: [ID 702911   local2.error] [MSG-70047] xxxxxxxxxxxxxxxxxxxxxxx 
   Feb  4 12:55:27 Someprocessname.py I2G(JV)-300[20448]: [ID 702911  local2.error] [MSG-70055] xxxxxxxxxxxxxxxxxxxxxxx
  </code></pre>
  
  <p>in addition to a lot of other gobblygook.</p>
  
  <p>Right now, it gets stuck in the for g in groups: </p>
  
  <p>I am relatively new to python and asynchronous programming. Ideally, I would like to be able to have the tail running parallely to the main process, and read new data with each loop. </p>
  
  <p>Please let me know if I need to add more information.</p>