v.25.9Improvement

Support writing multiple data files

Support writing multiple data files in Iceberg in a single insertion. Add new settings, iceberg_insert_max_rows_in_data_file and iceberg_insert_max_bytes_in_data_file to control the limits. #86275 (scanhex12).
Support for writing multiple data files in Iceberg tables during a single insertion with new configurable limits.

Why it matters

This feature allows users to control the size and number of data files created during insertions into Iceberg tables, enabling better management of file sizes and improving ingestion performance and storage efficiency.

How to use it

Users can enable and configure this feature by setting the new parameters iceberg_insert_max_rows_in_data_file and iceberg_insert_max_bytes_in_data_file to define the maximum number of rows and bytes per data file when inserting into Iceberg tables.