v.24.7
Backward Incompatible Changes
7- Forbid CRATE MATERIALIZED VIEW with Replicated databases
- clickhouse-keeper-client requires string literals for paths
- Metric KeeperOutstandingRequets Renamed to KeeperOutstandingRequests
- Remove is_deterministic Field from system.functions Table
- Function tuple Enhancements for Named Tuples in ClickHouse
- Change Deduplication Process for Materialized Views to Handle Duplicate Blocks Better
- Functions bitShiftLeft and bitShiftRight Return Error for Out of Bounds Shift Positions
Bug Fix
75- Check and Handle Cyclic Dependencies in Queries to Prevent Deadlocks
- Fix LowCardinality Column Size Issues in Function Calls
- Fix crash in maxIntersections
- Fix VALID UNTIL Clause Resetting After Restart
- Fix remaining time column in SHOW MERGES
- Fix duplicate "Query was cancelled" message in clickhouse-client
- Fixed crash in MaterializedMySQL with TABLE OVERRIDE for MySQL NULL to ClickHouse not NULL mapping
- Fix PREWHERE Logic Error for Tables without Adaptive Index Granularity
- Fix cancellation buffer bug in query cancellation
- Fix Metadata Handling for Filling Parts Columns
- Fix crash in ALTER TABLE SQL SECURITY on cluster
- Fix crash on AccessControl destruction with explicit shutdown
- Eliminate Injective Function in uniq* Arguments Recursively
- Fix CTE projection name issue in query
- Require dictGet Privilege for Dictionary Access in ClickHouse
- Fix S3 Authentication for User-Specific Incremental Backups
- Disable non-intersecting-parts optimization for FINAL queries with read-in-order enabled
- Fix Index Out of Bounds Exception for Filtered Blob Metadata
- Fix NOT_FOUND_COLUMN_IN_BLOCK in Deduplicate Merge of Projection
- Fixed MergeJoin Bug in Sparse Serialization Handling
- Fixed bug in compatibility level '23.4' application
- Fix ODBC Table to Support Nullable Fields
- Fix data race in TCPHandler during fatal errors
- Fix invalid exceptions in parseDateTime function with %F and %D placeholders
- Handling PostgreSQL Query Cancellation in ClickHouse
- Fix bug in short circuit logic with old analyzer and dictGetOrDefault
- Fix Bug Causing Corrupted SST Files in EmbeddedRocksDB with TTL
- Functions bitTest, bitTestAll, and bitTestAny Return Error for Out-of-Bounds Bit Index
- Setting join_any_take_last_row Support in Hash Join Queries
- Better handling of join conditions with IS NULL checks
- Fix Memory Leak in S3Queue
- Fix Tie Handling in arrayAUC to Align with Sklearn
- Fix TLS Connection Issues with MySQL Server Protocol
- Fix MySQL client protocol TLS connection issues
- Fix SSL_ERROR handling with zero timeout
- Add Missing Settings in Schema Inference Cache to Prevent Incorrect Results
- Column _size in s3 engine indicates file size, not archive size
- Fix dynamic subcolumn resolution in analyzer to optimize reading process
- Fix Config Merging for from_env with Replace Overrides
- Fix hanging issue in GRPCServer during shutdown
- Fixed Issues in has Function with LowCardinality Arguments
- Fix for groupArrayIntersect and deserialise behavior issues
- Fixed Buffer Overflow in unbin/unhex Implementation
- Disable merge-filters optimization due to exception risk
- Fixed Avro Parsing Issue with Negative Block Size Arrays in ClickHouse
- Fixed bug in ZooKeeper client preventing session from becoming unusable after hardware errors
- Fix SumIfToCountIfVisitor Issue with Signed Integers
- Fix rare missing data issue in distributed query results
- Fix parsing order of metadata fields in StorageDeltaLake
- Don't throw TIMEOUT_EXCEEDED for none_only_active mode in distributed_ddl_output_mode
- Fix limit handling for system.numbers_mt without usable index
- Fixed CPU Core Detection in ClickHouse for cgroups v2 in Containers
- Fix Not-ready Set Error for IN Subquery in Constraint
- Fix error reporting for S3 and AzureBlobStorage copying
- Prevent watchdog from retaining descriptors of unlinked log files
- Fix logical type loss in LogicalExpressionOptimizerPass
- Fix Column Identifier Registration Error with Group By Nulls and New Analyzer
- Fix Incorrect Results for Outer Joins with External Database due to Aggressive Filter Pushdown
- Added Column Materialization for Cross Join
- Fix Cannot find column error in GROUP BY with new analyzer enabled
- Avoid logical error during Npy import with bad array nesting and improve error testing
- Fix count() result with non-deterministic functions in predicate
- Correct memory tracking for Allocator::realloc
- Fix uninitialized memory issue when hashing empty tuples
- Fix Invalid Results for Parallel WINDOW Queries with Sparse Serialization
- Fix Named Collections Removal in Local Storage
- Fix column_length update in ColumnTuple::insertManyFrom
- Fix errors for queries with (column IS NULL) expression in ClickHouse
- Fix QUERY node error for scalar subquery in IN clause
- Fix PARAMETER_OUT_OF_BOUND Error in Variant Subcolumn Reading
- Fix rare stuck merge issue after dropping a column
- Fix isUniqTypes Assertion for Insert Select from Remote Sources
- Fix logical error in PrometheusRequestHandler (#66621)
- Fix indexHint function case identified by fuzzer
- Fix AST Formatting for 'create table b empty as a'
Experimental Feature
6- Change Binary Serialization of Variant Data Type to Include Compact Mode
- Support On-Disk Backend Storage for ClickHouse-Keeper
- Refactor JSONExtract Functions to Support More Types including Dynamic Type
- Support Null Map Subcolumn for Variant and Dynamic Subcolumns
- Fix Dynamic Subcolumn Reading from Altered Memory Table
- Add support for cluster_for_parallel_replicas with custom key in MergeTree tables
New Feature
15- Add ASOF JOIN Support for full_sorting_join Algorithm
- Support JWT Authentication in ClickHouse Client for Cloud
- Add SQL Functions for Date and Time Manipulation
- Introduce Startup Scripts for Preconfigured Query Execution
- Support for accept_invalid_certificate in client config for self-signed certificate connections
- Add system.error_log for error history from system.errors
- Add `groupConcat` Aggregate Function with Custom Delimiter and Element Limit
- Add AzureQueue Storage Support
- Add Setting to Control Writing Page Index in Parquet Files
- Introduce logger.console_log_level server config for console log control
- Automatically Append Wildcard to Directory Paths in File Table Function
- Add --memory-usage Option to Non-Interactive Client Mode
- Interactive Client for ClickHouse Disks with Local Directory Support
- Lightweight Delete Behavior in Tables with Projections
- Add System Tables for Detached Table Information
Improvement
38- Improved ZooKeeper Load Balancing with AZ-aware Support
- Optimize Trivial Insert Select Setting Disabled by Default
- Print Stacktrace and Diagnostics on ClickHouse Client Crashes
- Result of SHOW INDEX now sorted by primary key column position
- Change in Materialized Views Deduplication Implementation
- Support for Reading Partitioned DeltaLake Data and Inferring Schema from Metadata
- TLS Layer Update: Only certificateFile and privateKeyFile Parameters Accepted in Composable Protocols
- Added Profile Event for SELECT Queries Using Primary Key
- StorageS3Queue Enhancements: Default Values, Exception Handling, and Commit Controls
- Support aliases in parametrized view function for new analyzer
- Updated Azure Blob Storage to Mask Account Key in Logs
- Partition Pruning for IN Predicates in PARTITION BY Expressions
- arrayMin/arrayMax Applicable to All Comparable Data Types
- Improved Memory Accounting for cgroups v2 to Exclude Page Cache
- Do not create format settings for each row in EmbeddedRocksDB serialization
- Reduce ClickHouse Local Prompt to Smiley Face for macOS
- Avoid jemalloc messages on low-end virtual machines
- Disable Filesystem Cache Background Download by Default
- Add Config Option for ClickHouse Reload Interval
- Implement Binary Encoding for ClickHouse Data Types
- Server Settings Now Display `compiled_expression_cache_size` and `compiled_expression_cache_elements_size` in `system.server_settings`
- Add x509 SubjectAltName support for user identification
- clickhouse-local now respects memory usage settings from configuration file
- Add ClickHouse File Backup Script
- PostgreSQL Support for Query Cancellations
- Make allow_experimental_analyzer Controlled by Initiator for Distributed Queries
- Respect cgroup CPU limit in Keeper
- Allow concat function with empty arguments in ClickHouse
- Allow Control of Named Collections in clickhouse-local
- Improve Azure-Related Profile Events
- Support ORC File Reading with Writer's Time Zone
- Add PostgreSQL Connection Settings for Timeout and Retries
- Reduce inaccuracy of input_wait_elapsed_us/elapsed_us in system.processors_profile_log
- Improve ProfileEvents for Filesystem Cache
- Add settings to ignore ON CLUSTER clause for named collection management in replicated storage
- Function generateSnowflakeID Now Accepts Machine ID Parameter to Prevent Collisions
- Disable Ctrl+Z Suspension in Interactive Mode
- Add Primary Key Type Validation Option for Dictionaries
Performance Improvement
12- Replace int to string algorithm with faster method
- Sizes of hash tables cached for optimized query execution
- Optimized Query Performance with Buffered ORDER BY and High Selectivity Conditions
- Improve loading performance of plain_rewritable metadata
- Attaching tables on read-only disks optimizes resource usage
- Support minmax hyperrectangle for Set indices
- Unload Outdated Parts' Primary Index to Optimize Memory Usage
- Functions replaceRegexpAll and replaceRegexpOne Optimized for Trivial Patterns
- s3 Requests: Adjust Retry Times and Counts for Queries and Backups
- Support Query Plan LIMIT Optimization for PostgreSQL
- Improved ZooKeeper Load Balancing with AZ-Aware Support
- DatabaseCatalog Enhances Table Deletion Speed with Concurrency Threads