v.24.1New Feature

Implement New Object Storage Key Generation for S3 Disks with RE2 Regex Syntax

Official CH changelog text
Implementing the new way how object storage keys are generated for s3 disks. Now the format could be defined in terms of re2 regex syntax with key_template option in disc description. #57663 (Sema Checherinda).
Introduces a new flexible method for generating object storage keys for S3 disks in ClickHouse, allowing key formats to be defined using re2 regular expressions via the key_template option in disk configuration.

Why it matters

This feature provides greater control over how object keys are generated in S3 storage by enabling users to specify custom formats using regular expressions. It solves limitations of fixed key patterns, enhancing compatibility and customization for storage layout and retrieval.

How to use it

To use this feature, define the key_template option within the S3 disk description in ClickHouse's configuration. The key_template accepts a re2 regex pattern that specifies the desired key format for objects stored in S3.

Related resources

PreviousNext