v.21.2Improvements

Check Checksums of Distributed Batch Before Sending to Avoid INSERT Stuck Issues

Official CH changelog text
Check per-block checksum of the distributed batch on the sender before sending (without reading the file twice, the checksums will be verified while reading), this will avoid stuck of the INSERT on the receiver (on truncated .bin file on the sender). Avoid reading .bin files twice for batched INSERT (it was required to calculate rows/bytes to take squashing into account, now this information included into the header, backward compatible is preserved). #18853 (Azat Khuzhin).
Adds verification of per-block checksums on the sender side during distributed batch INSERTs to prevent hangs caused by truncated binary files, while avoiding reading .bin files twice.

Why it matters

This feature addresses the issue where INSERT operations on the receiver could get stuck due to truncated .bin files on the sender. By verifying block checksums without rereading files, it improves reliability and performance of distributed batch INSERTs. Additionally, it preserves backward compatibility by including necessary metadata in the header.

How to use it

The checksum verification is performed automatically during distributed batch INSERTs on the sender side. No special user action is required to enable this feature.

Related resources

PreviousNext