Why do you use the kafka client script written by python to produce data as soon as the program runs, but it takes a while to consume the data (there is data in the topic). (both pykafka and confluentKafka are the same)
there is only a very small probability that the data can be consumed immediately, and most of them have to wait a few minutes, which affects the efficiency of the test.
not for a few minutes. Can you post the code
https://stackoverflow.com/que. at the bottom of the answer, pay attention to the setting of the session.timeout.ms parameter in conf.
when you instantiate the consumer object, you add these two parameters, session_timeout_ms=6000,heartbeat_interval_ms=2000
.consumer = KafkaConsumer(self.kafkatopic, group_id = self.groupid,
bootstrap_servers = '{kafka_host}:{kafka_port}'.format(
kafka_host=self.kafkaHost,
kafka_port=self.kafkaPort
),
session_timeout_ms=6000,
heartbeat_interval_ms=2000)
there is an explanation of parameters on the official website python.readthedocs.io/en/master/apidoc/KafkaConsumer.html" rel=" nofollow noreferrer "> https://kafka-python.readthed.
is mainly this sentence:" If no heartbeats are received by the broker before the expiration of this session timeout, then the broker will remove this consumer from the group and initiate a rebalance. "
in addition, the default value of the local session_timeout_ms is 30s, not the 10s mentioned in the official website!
excuse me, have you solved your problem? When I consume kafka data, I traverse the messages and there is no response. Can you help me out?
Hello, have you solved your problem? I have the same problem as you. I changed the parameters but did not solve
.Previous: How to edit the field properties of a specified data table directly on web?
Next: How to make the scroll, in better-scroll package back to the top.