v.23.11Improvement

404 Response Issue for CompleteMultipartUpload Requires Reevaluation

404 response for CompleteMultipartUpload has to be rechecked. Operation could be done on server even if client got timeout or other network errors. The next retry of CompleteMultipartUpload receives 404 response. If the object key exists that operation is considered as successful. #56475 (Sema Checherinda).
ClickHouse S3 storage integration now treats a 404 response for the CompleteMultipartUpload operation as a successful completion if the object key exists.

Why it matters

This feature addresses an issue where network errors or client timeouts cause retries of CompleteMultipartUpload to receive a 404 response, despite the upload being completed on the server. By verifying the existence of the object key when a 404 is received, ClickHouse avoids failing the operation unnecessarily, improving reliability and consistency in uploads to S3-compatible storage.

How to use it

This behavior is applied automatically in the ClickHouse S3 storage engine when performing multipart uploads. Users do not need to enable any settings; the system will check for object existence upon receiving a 404 during CompleteMultipartUpload and treat the operation as successful if found.