v.25.9Improvement

Add rows/bytes limit for inserted data files

Add rows/bytes limit for inserted data files in delta lake. Controlled by settings delta_lake_insert_max_rows_in_data_file and delta_lake_insert_max_bytes_in_data_file. #86357 (Kseniia Sumarokova).
Added settings to limit the maximum number of rows and bytes per data file inserted into Delta Lake tables in ClickHouse.

Why it matters

This feature addresses the need to control the size of data files generated during inserts into Delta Lake, preventing excessively large files that can degrade performance and complicate data management. It helps users optimize file size for better query efficiency and maintainability.

How to use it

Users can configure the new settings delta_lake_insert_max_rows_in_data_file and delta_lake_insert_max_bytes_in_data_file to specify the maximum number of rows and bytes allowed in each inserted data file. Adjust these settings in the ClickHouse server configuration or per session to control data file size during insert operations.