Text index now supports the new split

Why it matters

This feature introduces the split tokenizer to enhance the flexibility of text indexing by allowing text to be tokenized based on custom delimiters. It helps users to efficiently parse and index text data according to specific splitting rules, improving search accuracy and performance.

How to use it

To use the split tokenizer, specify it in your text index settings when creating or modifying an index. Configure the tokenizer with the desired delimiter to control how text is split into tokens.