v.24.12
Backward Incompatible Changes
2Bug Fix
36- Fix Intersecting Parts Issue in MergeTree After Detached Directory Move Failure
- Fix Long Table Name Error Detection and Add Diagnostic Function
- Fixed Zombie Processes in ClickHouse Library Bridge After Crash
- Fix NoSuchKey Error in Transaction Rollback for plain_rewritable Disk
- Fix Dynamic Value Serialization in Pretty JSON Formats
- Add Inferred Format Name to Create Query for File, S3, URL, HDFS, Azure Engines
- Fix UDF Bugs in Join Expressions with Old Analyzer
- Fix small bugs in StorageObjectStorage and enable use_hive_partitioning by default
- Fix bug in min_age_to_force_merge_on_partition_only causing repeated merge attempts
- Fixed rare crash in SimpleSquashingChunksTransform for sparse columns
- Fixed Data Race in GraceHashJoin to Prevent Missing Rows
- Fixed ALTER DELETE Queries with Materialized _block_number Column
- Fixed data race in ColumnDynamic::dumpStructure during concurrent calls
- Fix LOGICAL_ERROR with Duplicate Columns in ORDER BY WITH FILL
- Fixed Type Mismatches After optimize_functions_to_subcolumns Application
- Use AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE Instead of AWS_CONTAINER_AUTHORIZATION_TOKEN_PATH
- Fix Parsing Errors for BACKUP DATABASE Queries
- Don't Allow Creating Empty Variant
- Fix Formatting of result_part_path in system.merges
- Fix glob parsing for single element cases
- Fix Query Generation for Follower Server with ARRAY JOIN in Distributed Queries
- Fix DateTime64 IN DateTime64 Bug
- Fixed Inconsistent Metadata for New Replicas in Replicated Databases
- Fix advanced SSL configuration for internal communication
- Fix S3Queue "No such key" Error with Smaller tracked_files_limit
- Fix exception in RemoteQueryExecutor for non-existent local users
- Fixed mutations with materialized _block_number column
- Fix backup/restore with rewritable disk for empty files
- Properly Cancel Inserts in DistributedAsyncInsertDirectoryQueue
- Fixed Crash in Sparse Column Data Parsing
- Fix crash during backup restore
- Fixed bug in parallel_hash JOIN method for complex ON clause conditions
- Use Default Format Settings for JSON Parsing to Prevent Deserialization Issues
- Fix crash in transactions with unsupported storage
- Fix Memory Tracking Overestimation Issue
- Check for Duplicate JSON Keys in Tuple Parsing
Build/Testing/Packaging Improvement
4Experimental Feature
1New Feature
14- Move JSON, Dynamic, and Variant types to beta and backport fixes to 24.11
- Schema Evolution in Iceberg Data Storage Format
- Integrate ClickHouse with Iceberg REST Catalog
- Added Cache for Primary Index in MergeTree Tables to Optimize Memory Usage
- Implement SYSTEM LOAD PRIMARY KEY Command for Efficient Index Loading
- Added support for attaching MergeTree tables as ReplicatedMergeTree and vice versa
- New Setting for Customizing HTTP Response Headers
- Add toUnixTimestamp64Second Function for DateTime64 Conversion to Int64
- Add "enforce_index_structure_match_on_partition_manipulation" Setting for Table Attachments
- Enhancements to ALTER USER and ALTER ROLE Syntax for Incremental Modifications
- Added arrayPRAUC function to calculate AUC for Precision Recall curve
- Add indexOfAssumeSorted Function to Optimize Search in Sorted Arrays
- Allows Delimiter as Optional Argument in groupConcat Function
- Function translate supports character deletion for mismatched argument lengths
Improvement
27- Remove allow_experimental_join_condition for Default Non-equi Conditions
- Server Config Settings Now Apply to Client
- Automatic Disk-Based GROUP BY/ORDER BY Based on Memory Usage
- Adding Cancellation Logic with CancellationChecker for Query Timeouts
- Support ALTER Migration from Object to JSON Type
- Allow Unknown Enum Values in Set
- Support LIKE Operator for Enum Data Type in ClickHouse
- Some ALTER USER Queries Were Accepted - Fixes #71227 and #71286
- Respect prefer_locahost_replica in distributed INSERT ... SELECT plans
- Azure's Iceberg Metadata Violations: Mislabeling v1 as v2 and Compatibility Issues
- Create Materialized View Now Supports UNION ALL with Insert Behavior Consistent with JOIN
- Added Source Query Validation for ClickHouse Dictionary
- Ensure ClickHouse Detects ZooKeeper Changes on Config Reloads
- Better memory usage approximation for cached marks in ClickHouse
- Add StartupScriptsExecutionState Metric for Tracking Execution Status
- Add MergeTreeIndexGranularityInternalArraysTotalSize Metric to system.metrics
- Add Retries for Creating Replicated Tables
- Add total_bytes_with_inactive to system.tables for inactive parts count
- Add MergeTree settings to system.settings_changes
- Support JSON Type in notEmpty Function
- Support Parsing GCS S3 Authentication Required Error
- Support Dynamic Type in ifNull and coalesce Functions
- Support Dynamic Type in toFloat64/touInt32 Functions
- Add S3 Request Settings for Backup and Restore Parsing
- Delete S3 Table Metadata After Last Usage
- Added JoinBuildTableRowCount, JoinProbeTableRowCount, and JoinResultRowCount Profile Events
- Support Subcolumns in MergeTree Sorting Key and Skip Indexes
Performance Improvement
16- JOIN Reordering: Selectable Inner Table in Query Plan
- Now using parallel_hash algorithm by default in join_algorithm setting
- Add Common Expression Extraction for JOIN Optimization
- Allows Index Usage on CASTed LowCardinality(String) Columns in SELECT Queries
- Improve Latency of Short Queries with Parallel Replicas by Offloading Index Analysis to Coordinator
- Memory Usage Optimized for ClickHouse Recursive Disk Removal
- Bring Back Optimization for Reading Subcolumns in Compact Parts
- Speed up sorting of LowCardinality(String) columns by de-virtualizing comparator calls
- Optimize argMin/argMax Functions for Simple Data Types
- Optimize Memory Tracker Locking for Improved Performance
- Add Async Executor Setting for Materialized Views
- Improved Performance in Deserialization of Aggregate Functions and RowBinary Parsing
- Split Ranges in Reading for Memory Efficiency with Parallel Replicas
- Speed up single partition key insertions in merge tree
- Implement Parallel Table Creation During Backup Restoration
- Dropping Mark Cache Can Cause Delays Due to Mutex Contention