title: Java Kafka consumer group failing to consume a few messages
body: <p>One issue is noticed that wherein Kafka consumer group (implemented in java) misses a few message from broker consistently. As a first line of debugging, via kafka console consumer, I can see those messages available in the broker.</p>
  
  <p>Kafka broker version: 0.10.1.0</p>
  
  <p>Kafka client version:</p>
  
  <pre><code>&lt;dependency&gt;
    &lt;groupId&gt;org.apache.kafka&lt;/groupId&gt;
    &lt;artifactId&gt;kafka_2.11&lt;/artifactId&gt;
    &lt;version&gt;0.9.0.1&lt;/version&gt;
  &lt;/dependency&gt;
  </code></pre>
  
  <p>Kafka consumer configuration:</p>
  
  <pre><code>Properties props = new Properties();
  props.put("bootstrap.servers","broker1,broker2,broker3");
  props.put("group.id", "myGroupIdForDemo");
  props.put("key.deserializer", StringDeserializer.class.getName());
  props.put("value.deserializer", StringDeserializer.class.getName());
  props.put("heartbeat.interval.ms", "25000"); 
  props.put("session.timeout.ms", "30000"); 
  props.put("max.poll.interval.ms", "300000");
  props.put("max.poll.records", "1");
  props.put("zookeeper.session.timeout.ms", "120000");
  props.put("zookeeper.sync.time.ms", "10000");
  props.put("auto.commit.enable", "false");
  props.put("auto.commit.interval.ms", "60000");
  props.put("auto.offset.reset", "earliest");
  props.put("consumer.timeout.ms", "-1");
  props.put("rebalance.max.retries", "20");
  props.put("rebalance.backoff.ms", "6000");
  
  KafkaConsumer&lt;String, String&gt; consumer = new KafkaConsumer&lt;&gt;(props);
  </code></pre>
  
  <p><strong>EDIT - Addition of some more information</strong></p>
  
  <p>Some more info I would like to add:
  There are total 6 partitions. However, total number of consumers are 40 for the topic having the same consumer group id. I do understand that 34 consumers sit idle and do nothing.</p>
  
  <p>However, the aspect I would like to understand is, if a consumer fails to send heartbeat to the extent that broker considers as dead and reassigns the partition, will any of the idle consumer get a chance to consume message ? This issue of message being not consumed is always being noticed in certain partitions only. I mean messages are failed to be delivered/consumed from the same partition.</p>
  
  <p>Any help is appreciated. Thanks.</p>