v.22.4Experimental Feature

Option to Store Parts Metadata in RocksDB to Speed Up ClickHouse Server Startup Time

Official CH changelog text
An option to store parts metadata in RocksDB. Speed up parts loading process of MergeTree to accelerate starting up of clickhouse-server. With this improvement, clickhouse-server was able to decrease starting up time from 75 minutes to 20 seconds, with 700k mergetree parts. #32928 (李扬).
Introduces an option to store MergeTree parts metadata in RocksDB, significantly speeding up the parts loading process and reducing the clickhouse-server startup time.

Why it matters

This feature addresses the problem of slow startup times for clickhouse-server when dealing with a large number of MergeTree parts. By storing parts metadata in RocksDB, the startup time is drastically reduced, improving operational efficiency and responsiveness for users managing extensive datasets.

How to use it

Enable the option to store parts metadata in RocksDB in the MergeTree configuration. This will accelerate the parts loading process during server startup and can be applied by modifying the relevant settings in the server configuration or table engine options.

Related resources

PreviousNext