v.19.14New Feature

Added Support for _partition and _timestamp Virtual Columns to Kafka Engine

Added support for _partition and _timestamp virtual columns to Kafka engine. #6400 (Ivan)
Added support for _partition and _timestamp virtual columns in the Kafka engine, allowing users to access Kafka message metadata directly within ClickHouse.

Why it matters

This feature enables users to retrieve Kafka partition and message timestamp information as virtual columns when querying Kafka tables in ClickHouse. It solves the problem of lacking direct access to these critical Kafka message attributes, enhancing data traceability and analysis.

How to use it

When creating or querying tables using the Kafka engine, users can reference the virtual columns _partition and _timestamp to access the Kafka message partition number and timestamp respectively. These columns are available automatically without additional configuration.