v.24.1New Feature

Add quantileDD Aggregate Function Using DDSketch

Add quantileDD aggregate function as well as the corresponding quantilesDD and medianDD. It is based on the DDSketch https://www.vldb.org/pvldb/vol12/p2195-masson.pdf. ### Documentation entry for user-facing changes. #56342 (Srikanth Chekuri).
Introduces the quantileDD aggregate function along with its variants quantilesDD and medianDD based on the DDSketch algorithm for approximate quantile estimation.

Why it matters

This feature provides an efficient and accurate method for estimating quantiles in large datasets with improved performance and memory usage, leveraging the DDSketch approach. It solves the problem of computing quantiles on massive data streams where exact calculation is costly or impractical.

How to use it

Use the aggregate functions quantileDD, quantilesDD, or medianDD in your SELECT queries to compute approximate quantiles. For example:

SELECT quantileDD(0.5)(value) FROM table;


This will return the approximate median of the value column.