v.20.12Improvement

Turn Off Parallel Parsing When Memory is Insufficient

Official CH changelog text
Turn off parallel parsing when there is no enough memory for all threads to work simultaneously. Also there could be exceptions like "Memory limit exceeded" when somebody will try to insert extremely huge rows (> min_chunk_bytes_for_parallel_parsing), because each piece to parse has to be independent set of strings (one or more). #16721 (Nikita Mikhaylov).
Disable parallel parsing in ClickHouse when available memory is insufficient for all parsing threads to operate concurrently, preventing memory exceptions during large row inserts.

Why it matters

This feature addresses issues where parallel parsing consumes more memory than available, which can lead to "Memory limit exceeded" errors, especially when inserting extremely large rows that exceed the min_chunk_bytes_for_parallel_parsing threshold. By turning off parallel parsing under low memory conditions, it ensures stable query execution and avoids crashes caused by memory overconsumption.

How to use it

ClickHouse automatically disables parallel parsing when it detects insufficient memory for all threads. Users do not need to manually enable or disable this feature. To benefit from this improvement, ensure that memory limits and parallel parsing settings are configured appropriately; the system will manage parallel parsing behavior based on available memory.

Related resources

PreviousNext