v.25.9Improvement
Support writing multiple data files
Support writing multiple data files in Iceberg in a single insertion. Add new settings,iceberg_insert_max_rows_in_data_fileandiceberg_insert_max_bytes_in_data_fileto control the limits. #86275 (scanhex12).
Why it matters
This feature allows users to control the size and number of data files created during insertions into Iceberg tables, enabling better management of file sizes and improving ingestion performance and storage efficiency.How to use it
Users can enable and configure this feature by setting the new parametersiceberg_insert_max_rows_in_data_file and iceberg_insert_max_bytes_in_data_file to define the maximum number of rows and bytes per data file when inserting into Iceberg tables.