v.24.5New Feature

Introduce Bulk Loading to EmbeddedRocksDB for Enhanced Import Speed

Official CH changelog text
Introduce bulk loading to EmbeddedRocksDB by creating and ingesting SST file instead of relying on rocksdb build-in memtable. This help to increase importing speed, especially for long-running insert query to StorageEmbeddedRocksDB tables. Also, introduce EmbeddedRocksDB table settings. #59163 #63324 (Duc Canh Le).
Introduce bulk loading to EmbeddedRocksDB by creating and ingesting SST files instead of relying on RocksDB built-in memtable, improving data import performance for StorageEmbeddedRocksDB tables. Additionally, new EmbeddedRocksDB table settings have been introduced.

Why it matters

This feature addresses the performance bottleneck during long-running insert queries on StorageEmbeddedRocksDB tables by enabling bulk loading using SST files. This method increases importing speed and efficiency compared to the traditional memtable-based insertion approach.

How to use it

Users can enable bulk loading on EmbeddedRocksDB tables by configuring the new EmbeddedRocksDB table settings introduced in this update. This involves adjusting table settings to allow SST file creation and ingestion for faster data import workflows.

Related resources

PreviousNext