v.23.8Improvement

Add S3 Request Throttling for Backup Operations

Official CH changelog text
Add the ability to throttle the S3 requests on backup operations (BACKUP and RESTORE commands now honor s3_max_[get/put]_[rps/burst]). #52974 (Daniel Pozo Escalona).
Introduced the ability to throttle S3 requests during BACKUP and RESTORE operations by honoring the s3_max_[get/put]_[rps/burst] settings.

Why it matters

This feature addresses the need to control the rate of S3 requests generated during backup and restore operations, helping to avoid hitting cloud provider rate limits, reduce cost spikes, and manage network usage more effectively.

How to use it

Users can enable throttling by configuring the parameters s3_max_get_rps, s3_max_get_burst, s3_max_put_rps, and s3_max_put_burst in the ClickHouse configuration. The BACKUP and RESTORE commands will then automatically honor these limits when accessing S3.

Related resources

PreviousNext