title: Spring AMQP RabbitMQ how to make two parallel consumers will not grab same task at the same time?
body: <p>I have two systems are integrated with RabbitMQ.</p>
  
  <p><hr/>
  <strong>BackGround</strong></p>
  
  <p>Client send multiple request messages from Spring-AMQP outbound-Gateway to RabbitMQExchange, the rabbitmq-<strong>DirectExchange</strong> will use <strong>round-robin</strong> dispatching those messages to Multiple Workers(Those workers are independent located on <strong>different desktops</strong> which will <strong>parallel</strong> running <strong>same worker code</strong> for processing <strong>different messages</strong> from RabbitExchange by use <strong>simpleMessageListner</strong>.</p>
  
  <hr/>
  
  <p><strong>Logic Flow</strong></p>
  
  <p>Similiar with <a href="http://www.rabbitmq.com/tutorials/tutorial-two-java.html" rel="nofollow">Rabbitmq Tutorial multiWorker-DirectExchange</a>.</p>
  
  <blockquote>
    <p><strong>Client</strong>-----sendRequests <strong>(5tasks)</strong> to ----><strong>RabbitMQ-DirectExchange</strong></p>
    
    <p>then <strong>Rabbitmq-DirectExchange</strong>  distribute those  5 tasks to workers
    <strong>PC1 ( Worker1 )</strong>, <strong>PC2 ( Worker2 )</strong></p>
  </blockquote>
  
  <p><hr/>
  <strong>ExchangeType &amp; my Bindings</strong>
  </p>
  
  <pre class="lang-html prettyprint-override"><code>&lt;!-- rabbit connection factory, rabbit template, and rabbit admin --&gt;
   &lt;rabbit:connection-factory
           id="connectionFactory"
           host="local IP address"
           username="guest"
           password="guest"
           channel-cache-size="10"   /&gt; 
  
   &lt;rabbit:template id="amqpTemplate" 
                    connection-factory="connectionFactory"
                    reply-timeout="600000" 
                    exchange="JobRequestDirectExchange"/&gt;
  
   &lt;rabbit:admin connection-factory="connectionFactory" id="rabbitAdmin" /&gt;
  
   &lt;rabbit:direct-exchange name="taskRequests" 
                           auto-delete="false" 
                           durable="true"  &gt;
       &lt;rabbit:bindings&gt;
          &lt;rabbit:binding queue="jobRequests" key="request.doTask"   /&gt;
       &lt;/rabbit:bindings&gt;
  
   &lt;/rabbit:direct-exchange&gt;
  
   &lt;rabbit:queue name="jobRequests" auto-delete="false" durable="true" /&gt;
  </code></pre>
  
  <hr/>
  
  <p><strong>Worker-The consumer Configuration</strong></p>
  
  <pre class="lang-html prettyprint-override"><code>&lt;rabbit:listener-container  id="workerContainer" 
          acknowledge="auto"
          prefetch="1"                         
          connection-factory="connectionFactory"&gt;
      &lt;rabbit:listener ref="taskWorker" queue-names="jobRequests" /&gt; 
  &lt;/rabbit:listener-container&gt;    
  </code></pre>
  
  <p></p>
  
  <p>The Worker Class is simple POJO who will process the request and complete task.</p>
  
  <hr/>
  
  <p>Use: <strong>RabbitMQ 3.2.2</strong> with <strong>Spring-Integration-Amqp 2.2</strong></p>
  
  <hr/>
  
  <p><strong>What I expect</strong></p>
  
  <p>I expect that <strong>Worker1</strong> can receive some of tasks while <strong>Worker2</strong> can pick the rest of tasks ( the other tasks ).</p>
  
  <p>I wish workers can <strong>parallel</strong> together do whole 5 tasks. Each time each worker <strong>only</strong> do one task, after finish will be distribute another tasks one by one. (<strong>rabbit-listner</strong> has been set to <strong>prefetch=1</strong>)</p>
  
  <p>Such as</p>
  
  <p>worker1: t2 t3 t5</p>
  
  <p>worker2: t1 t4</p>
  
  <p><strong>But</strong></p>
  
  <p>After lots of runtime-test, sometime it do the task correctly.</p>
  
  <p><strong>Worker1</strong>------task4 task1</p>
  
  <p><strong>Worker2</strong>------task3 task2 task5</p>
  
  <p>While sometime it do the <strong>wrong</strong> way like this:</p>
  
  <p><strong>Worker1</strong>------task4 task1</p>
  
  <p><strong>Worker2</strong>------task4 task2 task1</p>
  
  <p>Aparently, the task4 and task1 are be picked by worker1 and worker2 sametime.</p>
  
  <hr/>
  
  <h3>Runtime test:</h3>
  
  <p>I checked that the <strong>client</strong>  <strong>correctly</strong> send out task1 task2 task3 task4 task5 request message to RabbitExchange. But everytime each worker receive different tasks. There is a common case that may trigger wrong dispathcing.</p>
  
  <p>There are 5tasks <strong>(t1,t2,t3,t4,t5)</strong> at RabbitmqExchange, and they will be send to 2 parallel workers <strong>(w1,w2)</strong>.</p>
  
  <p><strong>w1</strong> got tasks:  <strong>t2 t1 t4</strong> </p>
  
  <p><strong>w2</strong> got tasks:  <strong>t3 t1</strong> </p>
  
  <p>As Round-Robin dispatch method, w1 and w2 in sequence got tasks. </p>
  
  <p><strong>w1</strong> got <strong>t2</strong> and <strong>w2</strong> got <strong>t3</strong>.</p>
  
  <p>While <strong>t2</strong> and <strong>t3</strong> running, RabbitmqExchange send <strong>t1</strong> to <strong>w1</strong> and wait for ack from <strong>w1</strong>. </p>
  
  <p>Suppose <strong>t2</strong> spend more time to finish task than <strong>t3</strong> does and <strong>w2</strong> is free when <strong>w1</strong> doing <strong>t1</strong>. </p>
  
  <p><strong>w2</strong> finish <strong>t3</strong> task will receive RabbitmqExchange dispatched <strong>t1</strong> because <strong>w2</strong> is not busy and RabbitExchange did not receive <strong>t1</strong> finished task ack message.</p>
  
  <p><strong>My understanding is</strong></p>
  
  <p>Both <strong>w1</strong> and <strong>w2</strong> are doing same task <strong>t1</strong>. Either one of them once finish <strong>t1</strong> will send back ack to RabbitmqExchange, then RabbitmqExchange will dequeue one task message. As <strong>t1</strong> has been finished <strong>twice</strong>, RabbitmqExchange dequeue one more message that it should. So in this way <strong>t5</strong> message has been dequeued because <strong>t1</strong> has been done twice. Although 5 messages in RabbitmqExchange are acked and dequeue finish. But two workers missing do <strong>t5</strong> and do <strong>t1</strong> twice.</p>
  
  <hr/>
  
  <h3>What should I do that can prevent two parallel workers grab the same message from a same Rabbit queue?</h3>
  
  <p>I <strong>tried</strong> auto-ack way, the message are correctly acked. But during the time of server wait for worker's ack, rabbitmq may redispatch the message which is not-acked but already been distributed to another worker.</p>
  
  <p>Also thinking about <strong>synchronized</strong> the sent out messages or <strong>give priority</strong> to sent out messages. But do not have clear vision how to accomplish.</p>
  
  <p>I am grateful to hear any ideas about this problem.Thanks</p>