v.24.4Improvement

Introduce Separate Consumer/Producer Tags for Kafka Configuration

Official CH changelog text
Introduce separate consumer/producer tags for the Kafka configuration. This avoids warnings from librdkafka (a bad C library with a lot of bugs) that consumer properties were specified for producer instances and vice versa (e.g. Configuration property session.timeout.ms is a consumer property and will be ignored by this producer instance). Closes: #58983. #58956 (Aleksandr Musorin).
Introduce separate consumer and producer configuration tags for Kafka integrations in ClickHouse to prevent cross-configuration warnings.

Why it matters

This feature addresses issues where consumer-specific properties were mistakenly applied to producer instances and vice versa, causing warnings from the underlying librdkafka library. By separating consumer and producer tags, configuration becomes clearer and warnings are eliminated, improving user experience and reliability.

How to use it

Configure Kafka settings in ClickHouse using distinct tags for consumer and producer properties. Assign consumer-related configurations under a consumer tag, and producer-related configurations under a producer tag to ensure appropriate settings are applied without warnings.

Related resources

PreviousNext