Consumer groups were invented to do parallel processing on partitions. Furthermore, they remove the need for consumers to be aware of partitions. The consumer group assigns partitions to the customers. ![[consumer-group-with-one-consumer.png]] If you are the only consumer in a consumer group, you will be responsible for all the partitions of the topic you subscribed to. For example, above consumer 1 subscribed to topic users, therefore it is now responsible for receiving messages from all of its partitions. ![[consumer-group-with-multiple-consumers.png]] When a new consumer is added to the group, the group rebalances the partition distribution among the available consumers in the group. **Each partition hast to be consumed by one and only one consumer. However, one consumer can consume many partitions. This is applicable to consumers that are in the same consumer group** For example, above you cannot have consumer 1 and consumer 2 consume the same partition (partition 1 or partition 2). However, if we only had one consumer, that consumer would be able to consume both partitions 1 and 2. * **To act like a queue, put all your consumers in one consumer group.** * In a queue system, each message can only be consumed once by a single consumer and then the message is deleted. * Putting all consumers in the same consumer group means that each partition can only be read by one and only one consumer like in a queue system. * The read position index of a partition is incremented sequentially after each read by default. Therefore, by default we cannot retrieve messages that we read in the past just like in a queue system. * Messages are not deleted like in queue system, therefore we can fix the read position index to go back and read past messages. * **To act like a pub/sub system, put each consumer in a unique consumer group** * **Partitions can be consumed by multiple consumers that are in different consumer groups.** * Topic partitions are group dependent. * **The outcome is that we get parallel processing for free.** * Consumers in the same consumer group can consume messages in parallel from different topic partitions. * Consumers in different consumer groups can consume messages from the same partition or different topic partition in parallel.