v.21.2Improvements

S3 Table Function Adds Support for Auto Compression Mode

Official CH changelog text
S3 table function now supports auto compression mode (autodetect). This closes #18754. #19793 (Vladimir Chebotarev).
The S3 table function in ClickHouse now supports an auto compression mode that automatically detects the compression type of files stored in S3.

Why it matters

This feature solves the problem of manually specifying compression types when querying compressed data from S3, simplifying the user experience and reducing errors. It allows ClickHouse to automatically detect and handle compressed data, improving usability and flexibility when working with diverse data sources in S3.

How to use it

When using the S3 table function, set the compression parameter to 'auto'. For example:

SELECT * FROM s3('https://bucket.s3.amazonaws.com/file', 'CSV', 'auto');

This enables automatic detection of the compression format.

Related resources

PreviousNext