v.24.1New Feature
Intel QuickAssist Technology Enhances ClickHouse with ZSTD_QAT Compression Codec
Intel QuickAssist Technology (QAT) provides hardware-accelerated compression and cryptograpy. ClickHouse got a new compression codec ZSTD_QAT which utilizes QAT for zstd compression. The codec uses Intel's QATlib and Inte's QAT ZSTD Plugin. Right now, only compression can be accelerated in hardware (a software fallback kicks in in case QAT could not be initialized), decompression always runs in software. #57509 (jasperzhu).Why it matters
This feature aims to improve the performance of data compression by utilizing Intel's hardware-accelerated QAT capabilities. It addresses the need for faster compression speeds in ClickHouse while maintaining compatibility through a software fallback when QAT is unavailable. Decompression is handled by software to ensure reliability.How to use it
Users can enable the hardware-accelerated compression by specifying theZSTD_QAT compression codec in table creation or data manipulation queries. If QAT hardware or drivers are not available, ClickHouse will automatically fall back to software compression. Decompression will always occur in software.