v.19.14New Feature

Function Hastoken(haystack, Token), Hastokencaseinsensitive(haystack, Token) to Check If Given Token Is in Haystack

Official CH changelog text
Function hasToken(haystack, token), hasTokenCaseInsensitive(haystack, token) to check if given token is in haystack. Token is a maximal length substring between two non alphanumeric ASCII characters (or boundaries of haystack). Token must be a constant string. Supported by tokenbf_v1 index specialization. #6596, #6662 (Vasily Nemkov)
Introduces the functions hasToken(haystack, token) and hasTokenCaseInsensitive(haystack, token) to check for the presence of a specific token within a string. A token is defined as a maximal length substring delimited by non-alphanumeric ASCII characters or string boundaries. These functions require the token to be a constant string and are supported by the tokenbf_v1 index specialization.

Why it matters

These functions allow efficient and precise detection of exact token occurrences within a string, especially useful for searching within text data where tokens are separated by punctuation or spaces. Leveraging the tokenbf_v1 index specialization significantly improves query performance by utilizing an index optimized for token-based lookups.

How to use it

Use hasToken(haystack, token) or hasTokenCaseInsensitive(haystack, token) in SELECT statements or WHERE clauses to check for the presence of tokens inside string columns. Ensure that the tokenbf_v1 index is created on the target column to benefit from index acceleration. The token argument must be specified as a constant string literal.

Related resources

PreviousNext