Enhanced parsing capabilities for data lakes by adding sequence ID parsing in manifest files and redesigning the Avro metadata parser for better extensibility.
Why it matters
This feature improves ClickHouse's ability to handle data lakes by enabling it to parse sequence identifiers from manifest files and provides a redesigned Avro metadata parser. This makes ClickHouse more adaptable to diverse data lake formats and facilitates future parser improvements, enhancing data ingestion reliability and flexibility.How to use it
Users can benefit from the enhanced parsing automatically when working with data lakes that include sequence IDs in manifest files or Avro metadata. No specific configuration is required, but users should ensure their data sources follow the supported formats to leverage these improvements.