title: How does offset work when I have multiple topics on one partition in Kafka?
body: <p>I am trying to develop a better understanding of how Kafka works. To keep things simple, currently I am running Kafka on one Zookeeper with 3 brokers and one partition with duplication factor of 3. I learned that, in general, it's better to have number of partitions ~= number of consumers. </p>
  
  <p><strong>Question 1</strong>: Do topics share offsets in the same partition?</p>
  
  <p>I have multiple topics (e.g. <code>dogs</code>, <code>cats</code>, <code>dinosaurs</code>) on one partition (e.g. partition 0). Now my producers have produced a message to each of the topics. <code>"msg: bark"</code> to <code>dogs</code>, <code>"msg: meow"</code> to <code>cats</code> and <code>"msg:  rawr"</code> to <code>dinosaurs</code>. I noticed that if I specify <code>dogs[0][0]</code>, I get back <code>bark</code> and if I do the same on <code>cats</code> and <code>dinosaurs</code>, I do get back each message respectively. This is an awesome feature but it contradicts with my understanding. I thought offset is specific to a partition. If I have pushed three messages into a partition sequentially. Shouldn't the messages be indexed with 0, 1, and 2? Now it seems me that offset is specific to a topic.</p>
  
  <p>This is how I imagined it</p>
  
  <pre><code>['bark', 'meow', 'rawr']
  </code></pre>
  
  <p>In reality, it looks like this</p>
  
  <pre><code>['bark']
  ['meow']
  ['rawr']
  </code></pre>
  
  <p>But that can't be it. There must be something keeping track of offset and the actual physical location of where the message is in the log file. </p>
  
  <p><strong>Question 2</strong>: How do you manage your messages if you were to have multiple partitions for one topic?</p>
  
  <p>In question 1, I have multiple topics in one partition, now let's say I have multiple partitions for one topic. For example, I have 4 partitions for the <code>dogs</code> topic and I have 100 messages to push to my Kafka cluster. Do I distribute the messages evenly across partitions like 25 goes in partition 1, 25 goes in partition 2 and so on...? </p>
  
  <p>If a consumer wants to consume all those 100 messages at once, he/she needs to hit all four partitions. How is this different from hitting 1 partition with 100 messages? Does network bandwidth impose a bottleneck? </p>
  
  <p>Thank you in advance</p>