v.23.8New Feature

Add CapnProto/Protobuf Schema Conversion Functions to ClickHouse

Official CH changelog text
Add new functions structureToCapnProtoSchema/structureToProtobufSchema that convert ClickHouse table structure to CapnProto/Protobuf format schema. Allow to input/output data in CapnProto/Protobuf format without external format schema using autogenerated schema from table structure (controlled by settings format_capn_proto_use_autogenerated_schema/format_protobuf_use_autogenerated_schema). Allow to export autogenerated schema while input/output using setting output_format_schema. #52278 (Kruglov Pavel).
Added new functions structureToCapnProtoSchema and structureToProtobufSchema for converting ClickHouse table structures to Cap'n Proto and Protobuf format schemas respectively, enabling native support for these formats without requiring external schema definitions.

Why it matters

This feature solves the problem of managing external format schemas when working with Cap'n Proto and Protobuf data. By automatically generating the schema from the ClickHouse table structure, it simplifies data input and output in these formats, reducing complexity and improving integration workflows.

How to use it

Use the functions structureToCapnProtoSchema or structureToProtobufSchema to generate the schema from a ClickHouse table structure. Enable automatic usage of the autogenerated schema for input/output by setting format_capn_proto_use_autogenerated_schema or format_protobuf_use_autogenerated_schema to 1. Additionally, use the output_format_schema setting to export the autogenerated schema during data processing.

Related resources

PreviousNext