v.21.4
Backward Incompatible Changes
7- toStartOfIntervalFunction Aligns Hour Intervals to Midnight
- Age and Precision Validation in Graphite Rollup Configs
- Fix significant subdomain detection for custom top-level domains in cutToFirstSignificantSubdomainCustom() and firstSignificantSubdomainCustom() functions
- Column keys replaced with key.names and key.types in system.dictionaries table
- ATTACH PARTITION Queries May Fail During Cluster Upgrade
- Rollback Issue with ClickHouse After ALTER ... ATTACH Query
- Empty remote_url_allow_hosts now blocks all remote host access in new version
Bug Fixes
41- Remove Socket from Epoll in HedgedConnections to Prevent Race Condition
- Add Memory Accounting to Parallel Parsing Routines to Prevent OOM
- Fix Exception with Digit-Named Columns in SELECT with Constant WHERE Condition
- Fix Query Cancellation with Hedged Requests and Async Socket Configuration
- Fix uncaught exception in InterserverIOHTTPHandler
- Fix Docker Entrypoint for Missing Http Port in Config
- Fix Invalid Number of Rows Error in JOIN with TOTALS and arrayJoin
- Fix Kafka Background Thread Pool Name to Ensure Message Consumption
- Fix query hanging for ReplicatedMergeTree during OPTIMIZE and ALTER operations
- Disable async_socket_for_remote and use_hedged_requests for buggy Linux kernels
- Docker Entrypoint: Prevent Chown of '.' When LOG_PATH Is Empty
- Fix Missing Size Check in AEAD Mode for Decrypt Function
- Merge Issue with CollapsingMergeTree Leading to Incomplete Granules Error
- Reversion of Change Preventing High Memory Usage in Hashed External Dictionaries
- Prevent hedged connections overlaps and fix Unknown packet 9 error
- Fix multipart/form-data handling in HTTP POST requests
- Fix ORDER BY Results with Window Functions in Query Optimization
- Fix deadlock in first catboost model execution
- Fix Query Result Error and Potential Crash in GROUP BY Conditions
- Better Error Handling and Logging in WriteBufferFromS3
- Fix Crashes in Aggregate Functions with Distinct During Two-Level Aggregation
- Fix Scalar Subquery Index Analysis (Related to Issues #21717 and #18896)
- Fix Decimal Column Type Bug in ReplicatedMerge Table Engines
- Fix infinite waiting in ReplicatedMergeTree during concurrent OPTIMIZE and DROP operations
- Fix arrayElement function for Map type with constant integer arguments
- Fix SIGSEGV for Non-Existing Attributes in ip_trie
- Server Connection Startup Now Requires DDLWorker and Dictionaries Initialization
- Add Type Conversion for Join Table Keys to Prevent SIGSEGV
- Fix Distributed Requests Cancellation with Async Socket for Remote
- Fix fsync_part_directory for horizontal merge
- Remove Unknown Columns from Joined Table in WHERE for External Database Queries
- std::terminate called on S3 write error
- Fix "Cannot find column" error with optimize_skip_unused_shards enabled and zero shards used
- Query may return incorrect empty result with constant WHERE condition and optimize_skip_unused_shards enabled
- Fix clusterAllReplicas function returning incorrect _shard_num
- Fix S3 Table Retaining Old Credentials After Config Update
- Fixed SSL Race Condition in Poco SecureSocket
- Fix Avro Format Parsing for Kafka
- Fix timeouts and non-blocking read in secure socket
- force_drop_table flag issue with MATERIALIZED VIEW fixed
- Fix name clashes in PredicateRewriteVisitor affecting WHERE filtration after full join
Build/Testing/Packaging Improvements
12- Add Jepsen Tests for ClickHouse Keeper
- Run Stateless Tests in Parallel in CI
- Enable Status Check for SQLancer CI Run
- PowerPC Build Improvements for ClickHouse
- Re-enable S3 library on aarch64
- Add tzdata to Docker containers for ORC format support
- Introduce Arguments for ClickHouse Dockerfile
- Allow clang-tidy with release builds by enabling assertions
- Add LLVM-12 Binaries and Update Build Compatibility in CMake 3.19
- Updating dockerd-entrypoint.sh for Yandex dockerhub-proxy due to Docker Hub rate limits
- Fix macOS Shared Library Build
- Add ctime option to zookeeper-dump-tree for node creation time display
Experimental Features
3Improvements
35- Add PostgreSQL Connection Pool to ClickHouse
- Support Non-Default Table Schema for Postgres Storage
- Support Replicas Priority for Postgres Dictionary Source
- Introduce min_bytes_to_rebalance_partition_over_jbod setting for balanced JBOD disk assignments
- Added Grant, Revoke, and System Values to query_kind in system.query_log
- Allow Independent Customization of HTTP Connection Timeouts for Replication
- Better Exception Messaging for Client During Server Block Writing Errors
- Fix Directory Already Exists Error in Temp Fetch Process
- Fix MSan Report for range Function with UInt256 Argument
- Add current_database column to system.processes table
- Add Case-Insensitive History Search and Subword Movement to ClickHouse Client
- Handling NULL Tuples in SQL IN Operator
- Update simdjson to version 0.9.1 to fix issue #21984
- Added Case Insensitive Aliases for CONNECTION_ID() and VERSION() Functions
- Add strict_increase option to windowFunnel function
- Enhance MergeTree Table to Expose DateTime64 Values in System Parts and Columns
- Supported replication_alter_partitions_sync in clickhouse-copier and decreased default timeouts
- Show Path to EmbeddedRocksDB Tables Data Directory in System Tables
- Add HedgedRequestsChangeReplica Event and Update Read Data Timeout Format
- DiskS3 Update: Fixed Directory Move Bug with Non-Empty Destinations
- Better formatting for Array and Map data types in Web UI
- Update clusters based on configuration changes
- Propagate Query and Session Settings for Distributed DDL Queries in ClickHouse
- Fix potential buffer overflow in PODArray instantiation with non-standard element sizes
- Add last_error_time, last_error_message, last_error_stacktrace, and remote columns to system.errors
- Add aliases simpleJSONExtract/simpleJSONHas to visitParam/visitParamExtract functions
- Add optimize_skip_unused_shards_limit Setting to Limit Sharding Key Values
- Improve clickhouse-format error handling for queries
- Improve Integer Key Support in Map Data Type
- MaterializeMySQL: Reconnect on Lost MySQL Connection
- Support More Cases for Rewriting CROSS JOIN to INNER JOIN
- Do not create empty parts on INSERT with optimize_on_insert enabled
- MaterializeMySQL: Add Minmax Skipping Index for _version Column
- Add --backslash option to clickhouse-format for line-ending backslashes
- ClickHouse: No LOGICAL_ERROR for Mutating Covered Parts
New Features
18- Extended DateTime64 Range Support and Improved DateTime Functions
- Added Kerberos Authentication Support for Preconfigured Users and HTTP Requests
- Add prefer_column_name_to_alias Setting for Enhanced Compatibility with Database Aliasing Rules
- Added dictGetChildren and dictGetDescendants Functions in ClickHouse
- Added executable_pool dictionary source and closed issue #14528
- Added dictionary table function resembling Dictionary engine
- Support Nullable Type for PolygonDictionary Attribute
- Functions dictGet and dictHas Use Current Database Name for DDL-Created Dictionaries
- Added dictGetOrNull function to return Null for missing keys
- Added Async Update and Nullable Support in Cache Dictionaries
- Support dictHas function for RangeHashedDictionary
- Add timezoneOf function and standardize timezone-related aliases
- Add Optional GRANTEES Clause to CREATE/ALTER USER Commands
- Add slowdowns_count column to system.clusters for hedged requests
- Add _partition_id Virtual Column and partitionID() Function for MergeTree Engines
- Add isIPAddressInRange function for CIDR network prefix testing
- Added SQL Command to Unfreeze Partitions in ClickHouse
- Supports Implicit Key Type Conversion for JOIN
Performance Improvements
9- Supported parallel formatting in clickhouse-local and other areas
- Support Parallel Parsing for CSVWithNames and TSVWithNames Formats
- Enable mmap IO for file ranges from 64 MiB for performance improvement
- Add Cache for Files Using min_bytes_to_use_mmap_io to Improve Performance
- Avoid Using Codec NONE for Data Compression
- Faster GROUP BY with Small max_rows_to_group_by and group_by_overflow_mode='any'
- Optimize FINAL Queries with PREWHERE Column Movement in ClickHouse
- Improved performance by replacing memcpy implementation
- Improve Aggregation Performance with Sorting Key Optimization