Support for Parquet Format in Kafka Tables

Why it matters

This feature enables users to consume messages in Parquet format directly from Kafka topics, simplifying integration with data pipelines that produce or exchange data using the Parquet file format. It enhances flexibility and efficiency for real-time data ingestion scenarios.

How to use it

To use this feature, when creating a Kafka table, specify the Parquet format as the message format in the table definition. For example:

CREATE TABLE kafka_table (
    ...columns...
) ENGINE = Kafka SETTINGS
    kafka_broker_list = 'broker:9092',
    kafka_topic_list = 'topic',
    kafka_group_name = 'group',
    kafka_format = 'Parquet';

This configures the table to read messages in Parquet format from the specified Kafka topic.