v.21.12New Features

Add Parallel Reading from Multiple Files with Glob Support in FROM INFILE Clause

Add support for parallel reading from multiple files and support globs in FROM INFILE clause. #30135 (Filatenkov Artur).
Added support for parallel reading from multiple files and enabled usage of glob patterns in the FROM INFILE clause.

Why it matters

This feature improves data ingestion performance by allowing ClickHouse to read multiple files concurrently and simplifies specifying multiple input files using glob patterns. It solves the problem of sequential file reading bottlenecks and manual file listing, enhancing efficiency and usability during data import.

How to use it

Use the FROM INFILE clause with glob patterns (e.g., FROM INFILE 'data/*.csv') to specify multiple files. ClickHouse will read these files in parallel automatically without additional configuration.