Added support in the AvroConfluent Kafka engine to skip tombstone and broken Kafka records during ingestion.
Why it matters
This feature addresses the issue of problematic Kafka records such as tombstone (delete marker) and corrupted messages that can disrupt data ingestion workflows. By skipping these records, it improves the robustness and stability of data pipelines using the AvroConfluent Kafka engine in ClickHouse.How to use it
Enable the option to skip tombstone and broken records in the AvroConfluent Kafka engine configuration when defining the Kafka table. This will cause the engine to automatically ignore these records during data ingestion.