{"action":"created","issue":{"url":"https://api.github.com/repos/klarna/brod/issues/182","repository_url":"https://api.github.com/repos/klarna/brod","labels_url":"https://api.github.com/repos/klarna/brod/issues/182/labels{/name}","comments_url":"https://api.github.com/repos/klarna/brod/issues/182/comments","events_url":"https://api.github.com/repos/klarna/brod/issues/182/events","html_url":"https://github.com/klarna/brod/issues/182","id":214418770,"number":182,"title":"Subscribing on brand new cluster failing","user":{"login":"meadoch1","id":375942,"avatar_url":"https://avatars3.githubusercontent.com/u/375942?v=3","gravatar_id":"","url":"https://api.github.com/users/meadoch1","html_url":"https://github.com/meadoch1","followers_url":"https://api.github.com/users/meadoch1/followers","following_url":"https://api.github.com/users/meadoch1/following{/other_user}","gists_url":"https://api.github.com/users/meadoch1/gists{/gist_id}","starred_url":"https://api.github.com/users/meadoch1/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/meadoch1/subscriptions","organizations_url":"https://api.github.com/users/meadoch1/orgs","repos_url":"https://api.github.com/users/meadoch1/repos","events_url":"https://api.github.com/users/meadoch1/events{/privacy}","received_events_url":"https://api.github.com/users/meadoch1/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":9,"created_at":"2017-03-15T15:03:36Z","updated_at":"2017-03-15T16:56:00Z","closed_at":null,"body":"I am having issues running unit tests against a brand new kafka/zookeeper cluster.  I pre-create the topics using command line tools but the first time I run the unit test against the cluster the subscribers never get the messages.  Whenever I run the test again it runs without error.  If I blow away the cluster, rebuild, and retry I get the same pattern of fail/pass (forever after).\r\n\r\nI am using Elixir so hopefully the code is readable enough to find whatever the bug is.\r\n\r\nTest Code\r\n```\r\ndefmodule MultiPass.Kafka.BrodTest do\r\n  use ExUnit.Case, async: false\r\n  doctest MultiPass.Kafka.Brod\r\n  alias MultiPass.Kafka.Brod\r\n\r\n  # @moduletag :functional\r\n  @topic1 \"testtopic1\"\r\n  @topic2 \"testtopic2\"\r\n\r\n  @delivery_timeout 10_000\r\n\r\n  setup do\r\n    {:ok, _} = Application.ensure_all_started(:brod)\r\n  end\r\n\r\n  describe \"sending and receiving messages\" do\r\n    test \"sends to all topics on a given group\" do\r\n\r\n      group1_id = random_key\r\n      group2_id = random_key\r\n      message1_key = random_key\r\n      message2_key = random_key\r\n\r\n      :ok = :brod.start_client([kafka: 9092], :client1, auto_start_producers: true)\r\n      group_config = [offset_commit_policy: :commit_to_kafka_v2,\r\n                      offset_commit_interval_seconds: 5,\r\n                      rejoin_delay_seconds: 2\r\n                     ]\r\n      consumer_config = [begin_offset: :earliest]\r\n      :brod.start_link_group_subscriber(:client1, \"group1\", [@topic1, @topic2], group_config, consumer_config, FakeClient, {self(), group1_id})\r\n      :brod.start_link_group_subscriber(:client1, \"group2\", [@topic1], group_config, consumer_config, FakeClient2, {self(), group2_id})\r\n\r\n      :brod.produce_sync(:client1, @topic1, 0, message1_key, \"This is a test\")\r\n      assert_receive({ group1_id, {:kafka_message, _offset, _, _, message1_key, \"This is a test\", _crc}}, @delivery_timeout)\r\n      assert_receive({ group2_id, {:kafka_message, _offset, _, _, message1_key, \"This is a test\", _crc}}, @delivery_timeout)\r\n    end\r\n  end\r\n\r\n  def random_key do\r\n    length = 10\r\n    :crypto.strong_rand_bytes(length) |> Base.url_encode64 |> binary_part(0, length)\r\n  end\r\n  \r\nend\r\n```\r\n\r\nTest output\r\n```\r\n 1) test sending and receiving messages sends to all topics on a given group (MultiPass.Kafka.BrodTest)\r\n     test/multi_pass/kafka/brod_test.exs:20\r\n     No message matching {group1_id, {:kafka_message, _offset, _, _, message1_key, \"This is a test\", _crc}} after 10000ms.\r\n     The process mailbox is empty.\r\n     stacktrace:\r\n       test/multi_pass/kafka/brod_test.exs:41: (test)\r\n\r\n     The following output was logged:\r\n\r\n     14:50:38.385 [info]  group coordinator (groupId=group1,memberId=,generation=0,pid=#PID<0.168.0>):\r\n     failed to join group\r\n     reason::GroupCoordinatorNotAvailable\r\n\r\n     14:50:38.385 [info]  group coordinator (groupId=group1,memberId=,generation=0,pid=#PID<0.168.0>):\r\n     re-joining group, reason::GroupCoordinatorNotAvailable\r\n\r\n     14:50:38.385 [info]  group coordinator (groupId=group2,memberId=,generation=0,pid=#PID<0.170.0>):\r\n     failed to join group\r\n     reason::GroupCoordinatorNotAvailable\r\n\r\n     14:50:38.385 [info]  group coordinator (groupId=group2,memberId=,generation=0,pid=#PID<0.170.0>):\r\n     re-joining group, reason::GroupCoordinatorNotAvailable\r\n\r\n     14:50:38.385 [info]  group coordinator (groupId=group1,memberId=,generation=0,pid=#PID<0.168.0>):\r\n     failed to join group\r\n     reason::GroupCoordinatorNotAvailable\r\n\r\n     14:50:38.385 [info]  group coordinator (groupId=group2,memberId=,generation=0,pid=#PID<0.170.0>):\r\n     failed to join group\r\n     reason::GroupCoordinatorNotAvailable\r\n\r\n     14:50:40.280 [info]  group coordinator (groupId=group1,memberId=,generation=0,pid=#PID<0.168.0>):\r\n     re-joining group, reason::GroupCoordinatorNotAvailable\r\n\r\n     14:50:40.290 [info]  group coordinator (groupId=group2,memberId=,generation=0,pid=#PID<0.170.0>):\r\n     re-joining group, reason::GroupCoordinatorNotAvailable\r\n\r\n     14:50:40.318 [info]  group coordinator (groupId=group1,memberId=,generation=0,pid=#PID<0.168.0>):\r\n     connected to group coordinator kafka:9092\r\n\r\n     14:50:40.318 [info]  group coordinator (groupId=group2,memberId=,generation=0,pid=#PID<0.170.0>):\r\n     connected to group coordinator kafka:9092\r\n\r\n     14:50:40.340 [info]  group coordinator (groupId=group2,memberId=nonode@nohost/<0.170.0>-d7e1f29f-5f7d-42dc-911c-780135f1d72c,generation=1,pid=#PID<0.170.0>):\r\n     elected=true\r\n\r\n     14:50:40.341 [info]  group coordinator (groupId=group1,memberId=nonode@nohost/<0.168.0>-ac25fd8f-1a57-4597-9085-73c4d58ff0ec,generation=1,pid=#PID<0.168.0>):\r\n     elected=true\r\n\r\n     14:50:40.437 [info]  group coordinator (groupId=group1,memberId=nonode@nohost/<0.168.0>-ac25fd8f-1a57-4597-9085-73c4d58ff0ec,generation=1,pid=#PID<0.168.0>):\r\n     assignments received:\r\n     testtopic1:\r\n         partition=0 begin_offset=undefined\r\n     testtopic2:\r\n         partition=0 begin_offset=undefined\r\n\r\n     14:50:40.438 [info]  group coordinator (groupId=group2,memberId=nonode@nohost/<0.170.0>-d7e1f29f-5f7d-42dc-911c-780135f1d72c,generation=1,pid=#PID<0.170.0>):\r\n     assignments received:\r\n     testtopic1:\r\n         partition=0 begin_offset=undefined\r\n\r\n     14:50:40.449 [info]  client :client1 connected to kafka:9092\r\n\r\n\r\n     14:50:48.028 [info]  group coordinator (groupId=group2,memberId=nonode@nohost/<0.170.0>-d7e1f29f-5f7d-42dc-911c-780135f1d72c,generation=1,pid=#PID<0.170.0>):\r\n     leaving group, reason :shutdown\r\n\r\n\r\n     14:50:48.028 [info]  group coordinator (groupId=group1,memberId=nonode@nohost/<0.168.0>-ac25fd8f-1a57-4597-9085-73c4d58ff0ec,generation=1,pid=#PID<0.168.0>):\r\n     leaving group, reason :shutdown\r\n```"},"comment":{"url":"https://api.github.com/repos/klarna/brod/issues/comments/286807521","html_url":"https://github.com/klarna/brod/issues/182#issuecomment-286807521","issue_url":"https://api.github.com/repos/klarna/brod/issues/182","id":286807521,"user":{"login":"meadoch1","id":375942,"avatar_url":"https://avatars3.githubusercontent.com/u/375942?v=3","gravatar_id":"","url":"https://api.github.com/users/meadoch1","html_url":"https://github.com/meadoch1","followers_url":"https://api.github.com/users/meadoch1/followers","following_url":"https://api.github.com/users/meadoch1/following{/other_user}","gists_url":"https://api.github.com/users/meadoch1/gists{/gist_id}","starred_url":"https://api.github.com/users/meadoch1/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/meadoch1/subscriptions","organizations_url":"https://api.github.com/users/meadoch1/orgs","repos_url":"https://api.github.com/users/meadoch1/repos","events_url":"https://api.github.com/users/meadoch1/events{/privacy}","received_events_url":"https://api.github.com/users/meadoch1/received_events","type":"User","site_admin":false},"created_at":"2017-03-15T16:56:00Z","updated_at":"2017-03-15T16:56:00Z","body":"Thanks @zmstone.  I'll try that and report back whether it fixes the issue."}}
{"actor":{"display_login":"meadoch1"}}