v.22.11Improvement
Increase S3 Multipart Upload Part Size to Avoid 10,000 Parts Limit Errors
Increase the size of upload part exponentially for backup to S3 to avoid errors about max 10 000 parts limit of the multipart upload to s3. #42833 (Vitaly Baranov).
Why it matters
This feature addresses the issue of hitting the maximum number of 10,000 parts allowed in S3 multipart uploads when backing up large datasets. By increasing the part size exponentially, it reduces the total number of parts, preventing upload failures and ensuring reliable backup operations.How to use it
The adjustment is applied automatically during backups to S3 and requires no user intervention. Users can continue to use theBACKUP command with S3 storage as usual, benefiting from improved multipart upload handling.