v.25.12Improvement

Add kafka_consumer_reschedule_ms as a tunable Kafka table

Add kafka_consumer_reschedule_ms as a tunable Kafka table engine setting in order to adjust how long consumers sleep for new data. Resolves #89204. #90112 (Jeremy Aguilon).
Added a new tunable setting kafka_consumer_reschedule_ms for the Kafka table engine in ClickHouse, allowing users to adjust the consumer sleep duration when waiting for new data.

Why it matters

This feature addresses the need to control how long Kafka consumers pause before polling for new messages, enabling better tuning of consumer responsiveness and resource usage according to workload characteristics.

How to use it

Users can set the kafka_consumer_reschedule_ms parameter when creating or altering a Kafka table engine to specify the desired sleep duration in milliseconds. For example:

CREATE TABLE kafka_example (
...
) ENGINE = Kafka
SETTINGS kafka_consumer_reschedule_ms = 500;

This setting controls how long the consumer waits before attempting to fetch new data from Kafka.