title: unique message check in kafka topic
body: <p>We use Logstash and we want read one table from Oracle Database and send these messages (as shown below) to Kafka:</p>
  
  <pre><code>Topic1: message1: {"name":"name-1", "id":"fbd89256-12gh-10og-etdgn1234njF", "site":"site-1", "time":"2019-07-30"}
          message2: {"name":"name-2", "id":"fbd89256-12gh-10og-etdgn1234njG", "site":"site-1", "time":"2019-07-30"}
          message3: {"name":"name-3", "id":"fbd89256-12gh-10og-etdgn1234njS", "site":"site-1", "time":"2019-07-30"}
          message4: {"name":"name-4", "id":"fbd89256-12gh-10og-etdgn1234njF", "site":"site-1", "time":"2019-07-30"}
  </code></pre>
  
  <p>Please note that <code>message1</code> and <code>message4</code> are the duplicates with the <code>same ID number</code>.</p>
  
  <p>Now, we want sure all messages are unique, so how can we filter <code>topic1</code> and unique all message then send to <code>topic2</code>?</p>
  
  <p>The end result we want:</p>
  
  <pre><code>Topic2: message1: {"name":"name-1", "id":"fbd89256-12gh-10og-etdgn1234njF", "site":"site-1", "time":"2019-07-30"}
          message2: {"name":"name-2", "id":"fbd89256-12gh-10og-etdgn1234njG", "site":"site-1", "time":"2019-07-30"}
          message3: {"name":"name-3", "id":"fbd89256-12gh-10og-etdgn1234njS", "site":"site-1", "time":"
  </code></pre>