Kafka multiple consumers have only one consumption

use the goalng "github.com/Shopify/sarama" library

I now have two consumers, and the two topic,kafka partitions are set to 50, but in the actual operation, only one consumer is consuming, closing the consumers who can currently consume, and the other can also consume, but it is not possible to run it at the same time

look at the documents on the Internet because there are not enough partitions, but now I have set 50 partitions, and there are only two consumers
code 1

.
    producerConfig := sarama.NewConfig()
    producerConfig.Producer.Partitioner = sarama.NewHashPartitioner
    producerConfig.Producer.Return.Successes = true
    producerConfig.Producer.Timeout = 5 * time.Second
    producer, err = sarama.NewSyncProducer([broker], producerConfig)


    kafka_msg := &sarama.ProducerMessage{
        Topic:topic,
        Key:sarama.StringEncoder(key),
        Value: sarama.StringEncoder(value),
    }
    partition, offset, err := producer.SendMessage(kafka_msg)

in this way, the partition id returned after successful delivery is always 0. I wonder if it is because my partition setting value does not take effect.

, but not sure because it is the first time to use kafka,

then use the next method
Code 2

    producerConfig := sarama.NewConfig()
    // producerConfig.Producer.Partitioner = sarama.NewHashPartitioner
    producerConfig.Producer.Return.Successes = true
    producerConfig.Producer.Timeout = 5 * time.Second
    producer, err = sarama.NewSyncProducer([broker], producerConfig)


    kafka_msg := &sarama.ProducerMessage{
        Topic:topic,
        Key:sarama.StringEncoder(key),
        Value: sarama.StringEncoder(value),
        Partition: 7 ,
    }
    partition, offset, err := producer.SendMessage(kafka_msg)

after this setting, the partition returns 0 for successful delivery.

what I want to make sure is that before the partition setting takes effect, if I write in code 2, will the final partition be delivered to 7?

I suspect now that the partition configuration does not take effect
Please answer

Apr.03,2021
There is a difference between unicast and broadcast in

kafka. For a message, there is a competitive relationship between consumers in the same consumer group, and only one consumer can consume, and this is unicast; similarly, for a message, consumers of different consumer groups can consume at the same time, this is multicasting. If you want both consumers to consume messages at the same time, you can put the two consumers in different consumption groups, which needs to be set by the groupId property of the consumer side.


A partition can only be consumed by one consumer in a consumer group. I guess you are here because each message sent is sent to the same partition


your two consumers should be in the same group. in kafka, a message can be consumed by multiple groups, but only by one consumer in a group.


In the

sarama package, when initializing producer, you need to specify the divider (partitioner).
stick a piece of source code:

unlike DefaultPatitioner, in Java producer, he selects partitions in order (specified partition, key hash, roundRobin).
so if you want to specify a partition.
will

producerConfig.Producer.Partitioner = sarama.NewHashPartitioner
Change

to

producerConfig.Producer.Partitioner = sarama.NewManualPartitioner
MySQL Query : SELECT * FROM `codeshelper`.`v9_news` WHERE status=99 AND catid='6' ORDER BY rand() LIMIT 5
MySQL Error : Disk full (/tmp/#sql-temptable-64f5-1e5136d-4c81e.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")
MySQL Errno : 1021
Message : Disk full (/tmp/#sql-temptable-64f5-1e5136d-4c81e.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")
Need Help?