v.23.10Improvement

Allow Skipping Null Values When Serializing Tuple to JSON

Official CH changelog text
Allow skipping null values when serailizing Tuple to json objects, which makes it possible to keep compatibility with Spark's to_json function, which is also useful for gluten. #55956 (李扬).
Added an option to skip null values when serializing Tuple types to JSON objects, enhancing compatibility with Spark's to_json function and improving integration with Gluten.

Why it matters

This feature solves serialization inconsistencies between ClickHouse and Spark by allowing users to omit null fields in JSON output. This ensures that JSON produced by ClickHouse matches Spark's expectations, facilitating smoother data interchange and improving overall usability for users employing Gluten.

How to use it

Users can enable skipping null values during Tuple JSON serialization by setting the appropriate serialization parameter or using the updated function interface that supports this behavior. This adjustment allows JSON outputs to exclude null fields, thereby aligning with Spark's to_json format.

Related resources

PreviousNext