v.25.12New Feature

Add a setting max_streams_for_files_processing_in_cluster_functions

Add a setting max_streams_for_files_processing_in_cluster_functions to control number of streams for parallel files reading in Cluster table functions. Closes #90223. #91323 (Pavel Kruglov).
Added the setting max_streams_for_files_processing_in_cluster_functions to control the number of parallel streams used for reading files in Cluster table functions.

Why it matters

This feature allows users to limit and manage the level of parallelism when processing files in Cluster table functions, helping to optimize resource usage and improve query performance on distributed file reads.

How to use it

Set the max_streams_for_files_processing_in_cluster_functions setting to the desired number of parallel streams before executing Cluster table functions. For example:

SET max_streams_for_files_processing_in_cluster_functions = 4;
SELECT * FROM cluster('cluster_name', 'database', 'table', 'file_function');