v.18.14Improvement
Kafka Engine Workflow Moved to Background Thread Pool for Load Management
The workflow of the Kafka engine has been moved to a background thread pool in order to automatically reduce the speed of data reading at high loads. Marek Vavruša.Why it matters
This feature aims to prevent overload and ensure stability by dynamically adjusting the data reading speed from Kafka when the system experiences high load, thereby improving performance and reliability for users ingesting data using theKafka engine.How to use it
No explicit user action is required to enable this feature. It is automatically applied by moving theKafka engine workflow to a background thread pool, which handles adjustment of the data ingestion speed under load.