v.20.4Improvement
Allow SAMPLE OFFSET Query for Splitting in ClickHouse Copier
Allow to useSAMPLE OFFSETquery instead ofcityHash64(PRIMARY KEY) % N == nfor splitting inclickhouse-copier. To use this feature, pass--experimental-use-sample-offset 1as a command line argument. #10414 (Nikita Mikhaylov)
Why it matters
This feature simplifies and standardizes the way data is partitioned during copying operations by enabling the use of theSAMPLE OFFSET query. It provides an easier and potentially more efficient method for splitting data across multiple workers compared to the previous hashing method.How to use it
To enable this feature, runclickhouse-copier with the command line argument --experimental-use-sample-offset 1. This activates the use of SAMPLE OFFSET splitting instead of the default hashing-based approach.