v.19.14New Feature

Support for Wildcards in Paths of Table Functions File and Hdfs

Support for wildcards in paths of table functions file and hdfs. If the path contains wildcards, the table will be readonly. Example of usage: select * from hdfs('hdfs://hdfs1:9000/some_dir/another_dir/*/file{0..9}{0..9}') and select * from file('some_dir/{some_file,another_file,yet_another}.tsv', 'TSV', 'value UInt32'). #6092 (Olga Khvostikova)
Added support for wildcards in paths of the file and hdfs table functions, enabling users to read data from multiple files matching a pattern in a single query.

Why it matters

This feature allows users to simplify querying multiple files or directories by using path wildcards instead of specifying each file explicitly, improving usability and flexibility when working with large datasets stored in file systems or HDFS. It provides a convenient way to read combined data while ensuring that the resulting table is readonly to maintain data consistency.

How to use it

Use wildcard patterns in the path argument of the file or hdfs table functions. Examples include:

SELECT  FROM hdfs('hdfs://hdfs1:9000/some_dir/another_dir//file{0..9}{0..9}')


and

SELECT * FROM file('some_dir/{some_file,another_file,yet_another}.tsv', 'TSV', 'value UInt32')


The resulting table will be readonly when using wildcards in the path.