v.25.7Improvement

Fix reading partitioned data

Official CH changelog text
Fix reading partitioned data in DeltaLakeCluster table function. In this PR cluster functions protocol version is increased, allowing to send extra info from initiator to replicas. This extra info contains delta-kernel transform expression, which is needed to parse partition columns (and some other staff in the future, like generated columns, etc). #82132 (Kseniia Sumarokova).
Fix reading partitioned data in the DeltaLakeCluster table function. This update increases the cluster functions protocol version to enable sending additional information from the initiator to replicas, specifically the delta-kernel transform expression needed to correctly parse partition columns and potentially other features like generated columns.

Why it matters

Previously, the DeltaLakeCluster table function faced issues correctly reading partitioned data. By allowing extra information such as the delta-kernel transform expression to be sent between cluster nodes, this feature ensures accurate parsing of partition columns and lays groundwork for supporting more complex data transformations. This improves reliability and correctness when querying partitioned Delta Lake data in a distributed ClickHouse environment.

How to use it

Users do not need to perform explicit actions to enable this fix. The cluster functions protocol version is automatically increased with this update, allowing the additional delta-kernel transform expression information to be sent transparently to replicas during query execution involving DeltaLakeCluster. To take advantage of this fix, simply upgrade to the version including this change and use the DeltaLakeCluster table function as usual.

Related resources

PreviousNext