v.21.6
Backward Incompatible Changes
3Bug Fix
80- Use Old Modulo Function Version for Partition and Primary Keys to Resolve Backward Incompatibility
- Fixed Infinite Processing of SYSTEM RESTART REPLICA and SYSTEM SYNC REPLICA Queries on Low RAM Servers
- Fix Monotonicity Issue in toWeek Function (Issue #24422)
- Fix drop partition with intersect fake parts issue
- Fixed Materialized View migration bug from Ordinary to Atomic database
- Allow Empty HTTP Headers in Client Requests
- Set max_threads = 1 to Resolve Memory Table Mutation Fail
- Fix typo in Memory tables implementation
- Fix Server Termination Due to HDFS Access Issues During Query Execution
- Fix crash when updating Nested column with const condition
- Fix Race Condition in RBAC Under Heavy Load
- Fix rare bug causing partially initialized tables to accept write requests
- Fix EXPLAIN PIPELINE issue with SELECT xxx FINAL showing wrong pipeline
- Fixed Comparison of const DateTime with DateTime64 in WHERE Clause
- Fix crash in merge JOIN, closes #24010
- Some ALTER PARTITION queries fixed to prevent intersection errors in replication queue
- Fix SIGSEGV for External GROUP BY with Overflow Row in ClickHouse
- Fix CACHE dictionary key metrics to address DictCacheKeysRequestedMiss overflows
- Fix PostgreSQL Connection Pool Implementation
- Fix distributed_group_by_no_merge = 2 with GROUP BY and Aggregate Functions
- Fix for s3 table function: Improved HTTP error handling
- Fix for s3 Table Function: Enhanced URI Handling for '+' Symbol
- Fix Initialization Error for GLOBAL IN/JOIN with use_hedged_requests
- Fix CLEAR COLUMN Issue with Materialized View
- Fix heap use after free in HDFS with Values format
- Avoid "Cannot schedule a task" Error on INSERT into Distributed
- Fixed bug in recovery of staled ReplicatedMergeTree replica affecting metadata updates during downtime
- Fix bug with Join and WITH TOTALS
- Fix Block Structure Mismatch Error in UNION Queries After Filter-Pushdown Optimization
- Add Type Conversion for optimize_skip_unused_shards_rewrite_in Setting
- Add check for updating nested subcolumns, resolves issue #22353
- Multiple Fixes for Hedged Requests in ClickHouse
- Fixed CSVWithNames Input Handling When Disabling Header Setting
- Fixed JDBC Bridge Timeout Connection Issue
- Fix logic for initial loading of complex_key_hashed with specified update_field
- Fixed crash with PREWHERE and row policy filter on empty result
- Avoid "Cannot schedule a task" error on INSERT into Distributed
- Added Exception for Identical Values in mannWhitneyUTest Function
- Fixed HTTP Data Insertion Exception Related to Server Fault
- Fixed LIKE Expression Misinterpretation with Escape Sequences
- Fixed Restart/Stop Command Hanging Issue
- Fixed COLUMNS matcher for multiple JOINs in select query
- Fixed crash when modifying default value of ReplacingMergeTree column
- Fixed corner cases in vertical merges with ReplacingMergeTree
- Fixed Bug with Casting Empty Array Literals to Multi-Dimensional Arrays
- Fixed deltaSum aggregate function bug after counter reset
- Fixed 'Cannot unlink file' error in ReplicatedMergeTree with multidisk configuration
- Fixed Incompatible Constant Expression Generation in Partition Pruning
- Fixed crash with 'auto' join_algorithm and Dictionary joins
- Don't Relax NOT Conditions During Partition Pruning
- Fixed rare race condition in background cleanup of old blocks
- Fixed rare race condition in ReplicatedMergeTree table management
- Fixed Key Dictionary Issue in DDL Creation
- Fixed ODBC Reading Issue with Long Column Names
- MaterializeMySQL: Fixed Not Found Column Error in Key Column Selection
- Correct Aliases Handling in Optimized Subqueries
- Server startup failure fixed when data_type_default_nullable is enabled in default profile
- Fixed Shutdown Crash Due to Incorrect Connection Accounting
- Fixed error when selecting from Materialized View after reattaching it to Atomic database
- Fix "Cannot find column in ActionsDAG result" Error Related to Untuple in Subqueries
- Fix Constant Columns of Type Map with Nullable Values
- Fixed formatDateTime and toDateTime64 for DateTime64 and large values
- Fixed Crash in mannWhitneyUTest and rankCorr with Window Functions
- LIVE VIEW: Fixed Hanging Issue in TEMPORARY LIVE VIEW Creation and Deletion
- Fixed HAVING Pushdown for Aggregated Filter Columns
- Fixed Zookeeper Request Hangs Due to OOM Exception
- Fixed Mutation Wait for ReplicatedMergeTree Table Engines
- Fixed Log Exception for Nested Types in SELECT Clause
- Fix infinite wait for auxiliary AWS requests
- Fixed Early Client Connection Crash in ClickHouse
- Map Data Type: Fix Incorrect Formatting of Map Function in Distributed Queries
- Fixed TSV Format Deserialization Issue for Empty Strings
- Fixed Column Cast in Merge Join Algorithm
- Buffer Overflow Vulnerability in tokenbf_v1 Full Text Index Resolved
- Do not limit HTTP chunk size; fixes #21907 and #22322.
- Fixed aggregation bug and improved performance with optimize_aggregation_in_order
- Check Usage of Table Function View as a Column
- Fix "unknown column" error for Merge engine with JOIN and aggregation
- Fixed Name Clashes in Pushdown Optimization for FULL JOIN WHERE Filtration
- Fixed rare quorum insert bug due to deduplication
Build/Testing/Packaging Improvement
22- Support for Building on Illumos and Solaris-Derived OSs
- Add Benchmarks for Hash Tables, Including Google's Swiss Table
- Update librdkafka to version 1.6.1
- Always Enable Asynchronous Unwind Tables for AArch64 Query Profiler Fix
- Avoid Build Dependency on Locale and Filesystem Order for Reproducible Builds
- Remove Nondeterminism for Byte-Identical Binaries
- Add Benchmarking Tool for ZooKeeper
- Run Stateless Tests in Parallel in CI
- Simplify Debian Packages to Fix Issues #21698 and #22976
- Added ClickHouse Support for Apple M1
- Fixed ClickHouse Keeper Build for macOS
- Fixed AArch64 Platform Tests
- Added function alignment for improved performance
- Adjust Tests for Consistent Results on amd64 and aarch64
- Allow Query Profiling Only on x86_64
- Allow Building with Unbundled xz Using CMake Option
- Enable bundled openldap on ppc64le
- Disable incompatible libraries on ppc64le
- Add Jepsen Test to CI for ClickHouse Keeper
- Build jemalloc with heap profiling support
- Avoid UB in Log Engines for rwlock Unlock
- Fixed UB by Unlocking rwlock in TinyLog from Same Thread
Experimental Feature
1Improvement
72- Add _partition_value Virtual Column to MergeTree for Deterministic Partition Pruning
- Added region parameter for S3 storage and disk
- Allow Configuring Different Log Levels for Logging Channels
- Keep Default Timezone on DateTime Operations
- Allow Empty String for MySQL Database Name to Use Default Database
- Fixed quantile(s)TDigest with singleton centroid handling and bug in over-compression of centroids
- Function now64 Supports Optional Timezone Argument
- Fix Progress Bar Overwriting Data in ClickHouse Client
- Fix rare crash due to memory allocation failure in simdjson
- Preserve dictionaries during storage shutdown to avoid errors
- Flush Buffer Tables Before Database Shutdown to Prevent Data Loss
- Now prefer_column_name_to_alias = 1 Favors Column Names in Group By, Having, and Order By
- Add ORDER BY WITH FILL Support for DateTime64
- Enable DateTime64 as Version Column in ReplacingMergeTree
- Log OS Name, Kernel Version, and CPU Architecture on Server Startup
- Support Specifying Table Schema for PostgreSQL Dictionary Source
- Add Suggestions for Enum Element Names on Typos
- Measure Found Rate for Dictionaries in ClickHouse
- Add RabbitMQ Queue Settings and Cleanup on Table Drop
- Add broken data file metrics to system.distribution_queue
- Querying system.tables no longer accesses ZooKeeper
- Respect lock_acquire_timeout_for_background_operations for OPTIMIZE queries
- Possibility to change S3 disk settings at runtime via SYSTEM RESTART DISK SQL command
- User Misconfiguring max_distributed_connections to Zero Triggers Incorrect Exception Message
- Disable min_bytes_to_use_mmap_io by Default
- Support LowCardinality nullability with join_use_nulls
- Added Restoration of MergeTree Parts to Detached Directory for S3 Disk
- Retries on HTTP Connection Drops in S3
- Add external_storage_max_read_rows settings for MySQL engine and MaterializeMySQL data fetches
- MaterializeMySQL Now Supports MySQL 5.7.9 with SQL Compatibility Fix
- Enable Subcolumn Reading for Distributed Tables
- Fix Tuple Usage in CREATE ... AS SELECT Queries
- Support for Parquet Format in Kafka Tables
- Increase max_uri_size to 1 MiB by default
- Set background_fetches_pool_size to 8 for improved production performance
- FlatDictionary Enhances Initial and Max Array Size Options
- Add non_replicated_deduplication_window setting for MergeTree inserts
- Update CatBoost Model Config Paths in Reloading
- Added Decimal256 Type Support in Dictionaries (Experimental Feature)
- Enabled async_socket_for_remote by default for distributed queries
- Fixed quantile(s)TDigest: Enhanced singleton handling and resolved centroid over-compression bug
- Make unhex function name case insensitive for MySQL compatibility
- Implement Generic Array Functions with Enhanced Type Support
- Raised max matches threshold in extractAllGroupsHorizontal function
- Do not optimize unused shards for single-node cluster
- Added SSL Support for ClickHouse-Keeper as Experimental ZooKeeper Replacement
- Added Background Buffer Flushing for Buffer Tables
- Fix rare exception when querying MergeTree table with NULL in WHERE condition
- Fix Error Handling in Poco HTTP Client for AWS
- Respect max_part_removal_threads for ReplicatedMergeTree
- Fix Corner Case in MergeTree Inactive Parts Settings
- dateDiff Enhancements for DateTime64 Support
- MaterializeMySQL Enhances MySQL Replication by Ignoring Views
- Allow RBAC Row Policy via PostgreSQL Protocol
- Add Metric for Buffer Layer Lock Wait Time Tracking
- Allow CTE in VIEW Definition
- Clear Terminal and Show Cursor in clickhouse-client
- Make round function behave consistently on non-x86_64 platforms with Banker's rounding
- Correct Structure Check for Data Blocks in Distributed Tables
- Allow Kafka Error Publishing to Virtual Column in ClickHouse
- Add Aliases for simpleJSONExtract and simpleJSONHas in visitParam Functions
- Add clickhouse-library-bridge for library dictionary source
- Forbid Dropping Columns Referenced by Materialized Views
- Support dynamic interserver credentials for zero-downtime rotation
- Add Kafka Storage Support for Arrow Formats
- Fixed Semicolon in Exception Message for Better Readability
- Fixed Whitespace in LowCardinality Exception Messages
- Some values no longer aligned center in Markdown table cells
- Remove non-essential details from suggestions in clickhouse-client
- Correct calculation of bytes_allocated for sparse_hashed dictionaries in system.dictionaries
- Fixed Total Rows Calculation for MergeTree Reverse Reading
- Fix infinite loop issue with self-referential ClickHouse dictionary configuration
New Feature
25- Add Postgres-like cast operator (::) for type conversion
- Make Big Integers Production Ready with Enhanced Support for Data Types and Functions
- Support Array Data Type for Arrow, Parquet, and ORC Formats
- Implement Table Comments, Closes #23225
- Support for Dictionary Management in clickhouse-local with DDL Queries
- Add uniqTheta Aggregate Function for Theta Sketch in ClickHouse
- Add splitByRegexp function
- Add arrayProduct function to return the product of array elements
- Add thread_name column to system.stack_trace
- Insert Default Values Instead of NULL in INSERT Queries with insert_null_as_default Option
- Add Progress Indication Support in clickhouse-local with --progress Option
- Add HTTP Compression Support in http Dictionary Source
- Added SYSTEM QUERY RELOAD MODEL and SYSTEM QUERY RELOAD MODELS
- Add JSON Output Setting for EXPLAIN PLAN Query
- Add 'indexes' Setting to 'EXPLAIN PIPELINE' Query for MergeTree Tables
- LDAP user DN detection for Active Directory group mapping in ClickHouse
- New deltaSumTimestamp Aggregate Function for Ordered Row Summation
- Added Less Secure IMDS Credentials Provider for S3 in Docker
- Add Back indexHint Function to Fix Issues #21238 and #9540
- Added dictGetChildren and dictGetDescendants Functions with Performance Improvements
- Added dictGetOrNull function to return Null for missing keys
- Added s3Cluster Function for Parallel File Processing on S3 in ClickHouse
- Added Support for Replicas and Shards in MySQL/PostgreSQL Table Engine
- Added ALTER TABLE ... FETCH PART for Single Part Retrieval
- Added max_distributed_depth Setting for Recursive Queries on Distributed Tables
Performance Improvement
15- Enable compile_expressions setting by default for native code compilation with LLVM
- Update re2 Library for Improved Performance and GCC-11 Compatibility
- ORC Input Format Optimization for Memory Efficiency in Stripe Reading
- Fusion of Sum, Count, and Avg Functions in Query Optimization
- Update zstd to v1.5.0 with improved compression performance
- Improved Buffer Performance by Eliminating Lock for Total Bytes/Rows
- Preallocate Support for Hashed/Sparse_Hashed Dictionaries
- Enable async_socket_for_remote by default to optimize Distributed tables querying
- Improved intDiv Performance with Dynamic Dispatch for AVX2
- Improved ArrowStream Input Performance for Non-Local Sources
- Disabled Default Compression for Localhost Interactions in ClickHouse
- Exclude Unused Shard Values from IN Clause in Distributed Queries
- Improved Performance for Column Subset Reading with File-like Table Engine
- Allow More Conditions in PREWHERE for Improved Performance
- Improved ODBC Performance and Support for New Data Types