title: Plot a continuous graph of Number of Snort alerts against time
body: <p>I have snort logging DDOS alerts to file; I use Syslog-ng to parse the logs and output in json format into redis (wanted to set it up as a buffer, I use 'setex' command with expiry of 70 secs). </p>
  
  <p>The whole thing seems not to be working well; any ideas to make it easier is welcome. </p>
  
  <p>I wrote a simple python script to listen to redis KA events and count the number of snort alerts per second. I tried creating two other threads; one to retrieve the json-formatted alerts from snort and the second to count the alerts. The third is supposed to plot a graph using matplotlib.pyplot</p>
  
  <pre class="lang-py prettyprint-override"><code>
  #import time 
  from redis import StrictRedis as sr 
  import os
  import json
  import matplotlib.pyplot as plt
  import threading as th
  import time
  
  
  redis = sr(host='localhost', port = 6379, decode_responses = True)
  
  
  #file = open('/home/lucidvis/vis_app_py/log.json','w+')
  
  # This function is still being worked on
  def  do_plot():
      print('do_plot loop running')
      while accumulated_data:
  
          x_values = [int(x['time_count']) for x in accumulated_data]
          y_values = [y['date'] for y in accumulated_data]
  
          plt.title('Attacks Alerts per time period')
  
          plt.xlabel('Time', fontsize=14)
          plt.ylabel('Snort Alerts/sec')
  
          plt.tick_params(axis='both', labelsize=14)
  
          plt.plot(y_values,x_values, linewidth=5)
          plt.show()
          time.sleep(0.01)
  
  
  
  
  def accumulator():
      # first of, check the current json data and see if its 'sec' value is same 
      #that is the last in the accumulated data list
      #if it is the same, increase time_count by one else pop that value
      pointer_data = {}
  
      print('accumulator loop running')
  
      while True: 
          # pointer data is the current sec of json data used for comparison
          #new_data is the latest json formatted alert received
          # received_from_redis is a list declared in the main function
          if received_from_redis:
              new_data = received_from_redis.pop(0)
          if not pointer_data:
              pointer_data = new_data.copy()
  
          print("&gt;&gt;", type(pointer_data), " &gt;&gt; ", pointer_data)
  
          if pointer_data and pointer_data['sec']==new_data["sec"]
              pointer_data['time_count'] +=1
  
  
          elif pointer_data: 
              accumulated_data.append(pointer_data)
              pointer_data = new_data.copy()
              pointer_data.setdefault('time_count',1)
  
          else:
              time.sleep(0.01)
  
  
  
  
  # main function creates the redis object and receives messages based on events
  #this function calls two other functions and creates threads so they appear to run concurrently
  
  def main():
      p = redis.pubsub()
      # 
      p.psubscribe('__keyspace@0__*')
  
      print('Starting message loop')
  
      while True:
          try:
              time.sleep(2)
              message = p.get_message()
  
      # Obtain the key from the redis emmitted event if the event is a set event
              if message and message['data']=='set':
           # the format emmited by redis is in a dict form
           # the key is the value to the key 'channel'
           # The key is in '__keyspace@0__*' form
           # obtain the last field of the list returned by split function
                  key = message['channel'].split('__:')[-1]
  
                  data_redis = json.loads(redis.get(str(key)))
                  received_from_redis.append(data_redis)
          except Exception e:
              print(e)
  
              continue
  
  
  
  
  if __name__ == "__main__":
      accumulated_data = []
      received_from_redis = []
  # main function creates the redis object and receives messages based on events
  #this function calls two other functions and creates threads so they appear to run concurrently
      thread_accumulator = th.Thread(target = accumulator, name ='accumulator')
      do_plot_thread = th.Thread(target = do_plot, name ='do_plot')
  
      while True:
          thread_accumulator.start()
          do_plot_thread.start()
  
          main()
  
      thread_accumulator.join()
      do_plot_thread.join()
  
  
  
  
  
  
  
  
  </code></pre>
  
  <p>I currently do get errors per se ; I just cant tell if the threads are created or are working well. I need ideas to make things work better.</p>
  
  <p>sample of the alert formated in json and obtained from redis below</p>
  
  <pre><code>
  {"victim_port":"","victim":"192.168.204.130","protocol":"ICMP","msg":"Ping_Flood_Attack_Detected","key":"1000","date":"06/01-09:26:13","attacker_port":"","attacker":"192.168.30.129","sec":"13"}
  
  </code></pre>