title: Fault tolerance in Kafka Cluster using Spring Boot
body: <p>I started work with Spring Boot Kafka and kafka cluster on docker. I've faced the problem when I started to check fault tolerance. Generally I created one topic (replication factor = 2) through Kafka Admin in Spring Boot. I was able to store some messages and consume them from Kafka. But after stopping one of the kafka container (I have two kafka instances and one zookeeper) my consumer was not able to communicate which remaining one.  </p>
  
  <p>Already I provide to bootstrap.servers property:</p>
  
  <pre><code>SPRING_KAFKA_BOOTSTRAP_SERVERS: kafka:9092,kafka2:9093
  </code></pre>
  
  <p>My kafka admin bean which create topic looks like:</p>
  
  <pre><code>@Bean
  public NewTopic adviceTopic() {
      return new NewTopic(topicName, 1, (short) 2);
  }
  </code></pre>
  
  <p>When I'm using only one instance (it's doesn't matter which one I choose) it works well (so it's not case of wrong configuration kafka listeners/host). When I'm removing one instance my application is not able to use the second one kafka instance.</p>
  
  <p>My docker compose looks like:</p>
  
  <pre><code>version: '2'
  services:
    zookeeper:
      image: wurstmeister/zookeeper
      ports:
      - "2181:2181"
    kafka:
      container_name: kafka
      image: wurstmeister/kafka
      expose:
      - "9092"
      environment:
        KAFKA_LISTENERS: PLAINTEXT://kafka:9092
        KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
        KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'false'
      volumes:
        - /var/run/docker.sock:/var/run/docker.sock
      depends_on:
        - zookeeper
    kafka2:
      container_name: kafka2
      image: wurstmeister/kafka
      expose:
        - "9093"
      environment:
        KAFKA_LISTENERS: PLAINTEXT://kafka2:9093
        KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
        KAFKA_AUTO_CREATE_TOPICS_ENABLE:
      volumes:
        - /var/run/docker.sock:/var/run/docker.sock
      depends_on:
        - zookeeper
    kafka-service:
       container_name: kafka_service
       build: .
       ports:
       - "8080:8080"
       environment:
         SPRING_KAFKA_BOOTSTRAP_SERVERS: kafka:9092,kafka2:9093
       depends_on:
         - zookeeper
         - kafka
         - kafka2
  </code></pre>
  
  <p>The result I've got when I used "docker stop kafka_container":</p>
  
  <p>2019-07-19 09:19:16.836  WARN 1 --- [| consumers] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=json-0, groupId=qpm-consumers] Connection to node 1002 could not be established. Broker may not be available.</p>
  
  <p>And after that I was not able to produce any messages using my Spring Boot Example. Could you provide me some tips, why my app was not able to use second one kafka instance? Even if I set properly replication factor?</p>