v.25.9Improvement
Add rows/bytes limit for inserted data files
Add rows/bytes limit for inserted data files in delta lake. Controlled by settingsdelta_lake_insert_max_rows_in_data_fileanddelta_lake_insert_max_bytes_in_data_file. #86357 (Kseniia Sumarokova).
Why it matters
This feature addresses the need to control the size of data files generated during inserts into Delta Lake, preventing excessively large files that can degrade performance and complicate data management. It helps users optimize file size for better query efficiency and maintainability.How to use it
Users can configure the new settingsdelta_lake_insert_max_rows_in_data_file and delta_lake_insert_max_bytes_in_data_file to specify the maximum number of rows and bytes allowed in each inserted data file. Adjust these settings in the ClickHouse server configuration or per session to control data file size during insert operations.