v.20.10
Backward Incompatible Changes
5- Make multiple_joins_rewriter_version Obsolete
- Change Default Value of format_regexp_escaping_rule to Raw for Improved User Expectations
- Add Support for Nested Multiline Comments in SQL
- Added MergeTree Settings for Controlling TTL Merges with Compatibility Considerations
- Upgrading ClickHouse: Preventing 'Part Intersects Previous Part' Errors During Rolling Updates
Bug Fix
124- Fix Memory Overallocation Issue Closing #14560
- Fix executable dictionary source hang in specific formats
- Fix double free in dictGet function on dictionary load error
- Fix Group By with Totals, Rollup, Cube Modifiers, and Min/Max Functions
- Fix Async Distributed INSERT with Localhost Replica Configuration
- Fix memory leak in TwoLevelStringHashTable implementation
- Fix segfault in lambdas due to wrong aggregation
- Fix query hang for ReplicatedVersionedCollapsingMergeTree with ALTER MODIFY ORDER BY
- MaterializeMySQL: Fix Collate and Charset Name Parser, Support Length = 0 for String Type
- Allow direct layout for dictionaries with complex keys
- Prevent Replica Hang Due to Inactivity Following Replication Errors
- Fix rare segfaults in MaterializedView with concurrent table drops
- Fix Ambiguity in Settings Profiles Parsing for CREATE USER Command
- MaterializeMySQL: Fix Crash on Create Database Failure
- Fixed Concurrent DDL Issues with DROP and RENAME Commands in Atomic Database Engine
- Fix empty result issue for Distributed table queries with WHERE, PREWHERE, and GLOBAL IN
- Fixes difference expressions with same alias during query reanalysis
- Fix rare deadlocks in RBAC implementation
- Fix Block Structure Mismatch in SELECT ORDER BY DESC Queries After ALTER MODIFY COLUMN
- MaterializeMySQL: Fix select count() inaccuracy
- Fix for Not Found Column Exception in Virtual Column Queries
- Fix Materialized View Drop Issue in Atomic Database
- Possibility to move part to another disk/volume after failed attempt
- Fix "Cannot find column" Error During MATERIALIZED VIEW Insertion with ARRAY JOIN
- Fixed max_replicated_logs_to_keep Setting and Improved Lost Replica Recovery Process
- Fix rare race condition in MySQL dictionaries and tables
- Fix benign race condition in AMQP-CPP
- Fix "Cannot add simple transform to empty Pipe" Error during Buffer Table Read
- Proper Error Handling for Insert into MergeTree with S3
- Fixed S3 Table Function Bug in ClickHouse
- Fix Resource Destruction Order in ReadFromStorage Step to Prevent Crashes
- Subtract ReadonlyReplica Metric on Detaching Readonly Tables
- Fixed "Element ... is not a constant expression" error in JSON function use with VALUES, LIMIT, or IN operator
- Query Optimization: Faster Completion on Exceptions
- Prevent Disk Space Calculation Error (errno: 4)
- Fix Database Not Found Error in Queries with IN and Distributed Table
- Mutation Fixes Hanging Issue After Partition Operations
- Fix ILIKE Case Sensitivity Bug After LIKE Execution
- Fix Missing Columns Errors in Data Selection
- Throw Error for Single Parameter in ReplicatedMergeTree
- Fix event subscription bug in DDLWorker causing query hangs in ON CLUSTER
- Report Error for Invalid Type in boundingRatio Aggregate Function
- Fix MySQL Engine Database Attachment Exception
- Fix handling of multiple column transformers in select queries
- Fixed S3 Storage Compression Issue
- Fix bug with empty time_zone argument in toStartOfDay queries
- Fix race condition in MergeTree table rename and cleanup
- Fix rare race condition on server startup with system logs enabled
- Fix MySQL query hang issue with multiple subqueries to the same table
- Fix Uninitialized Memory Issue in QueryLog's memory_usage Field
- Fix GROUP BY 'Unknown identifier' Error with JOIN on Merge Table
- Fix Instance Crash with joinGet on LowCardinality Types
- Fix Buffer Engine Bug Preventing Data Insertion After ALTER Query
- Adjust Decimal Field Size in MySQL Column Definition
- Fix Data Compression Issues and Retain LowCardinality Type in Join Algorithms
- Update jemalloc to Fix percpu_arena with Affinity Mask
- Fix FixedString Primary Key Comparison Logic in ClickHouse
- Buffer Overflow Vulnerability in Function Bar Resolved
- Fixed Invalid Argument Error on DDL Query Execution in ClickHouse on Docker for Mac OS
- Fix crash in RIGHT or FULL JOIN when memory limit exceeded
- Now settings can equal background_pool_size
- Fix Predicate Push Down for Subqueries with finalizeAggregation Function
- Publish CPU Frequencies per Logical Core in system.asynchronous_metrics
- MaterializeMySQL: Fixed .metadata.tmp File Exists Error
- Fix memory limit error in extractAllGroups function
- Fix SIGSEGV in StorageFile INSERT Operation
- Fixed segfault in cache dictionary #14837 #14879 by Nikita Mikhaylov
- MaterializeMySQL: Fix for MySQL binlog event parsing errors
- Fix rare error in SELECT queries with dependent DEFAULT expressions
- Fix Server Startup Issue with ZooKeeper Configuration Fetching
- Fix Monotonicity Detection Bug in Int to Int Cast for Signed Types
- Replace Column Transformer Updates for Cloned ASTs Fixing Issue #14695
- Fixed default database name in materialized view metadata during query modification
- Fix bug in ALTER UPDATE mutation with Nullable column leading to incorrect values or segfault
- Fix Decimal Multiplication Result Scale Issue
- Fix has function for LowCardinality of Nullable
- Cleanup Data Directory After Zookeeper Exceptions in StorageReplicatedMergeTree Engine
- Fix rare segfaults in -Resample function due to overflow with large parameters
- Fix Nullable(String) to Enum Conversion Bug
- Fixed Sorting Order of Nullable Column
- Fix currentDatabase() function for ON CLUSTER DDL query
- MaterializeMySQL: Fix for Packet Payload Not Fully Read Error
- Fix Silent Crashes in ClickHouse on Specific glibc Versions
- Fix IN operator for multiple columns with transform_null_in enabled
- Fix for optimize_read_in_order and optimize_aggregation_in_order with max_threads>0 in ORDER BY
- Fix LowCardinality Type Parsing in AVRO Input
- Fix Metadata Growth in MySQL to ClickHouse Integration with GTID Shrinking
- Fix DROP TABLE for Distributed (racy with INSERT)
- Fix Large Entry Processing in Replication Queue
- Fix MySQL Query Handling for Disabled ServerηΆζ
- Fixed issues with clickhouse-odbc-bridge and ODBC dictionary updates on dual IPv4/IPv6 stacks
- Fix Enum and Int Key Comparison Issue #17989
- Fixed Unique Key Convert Crash in MaterializeMySQL Engine
- Fixed std::out_of_range in S3 URL Parsing
- Fixed MySQL to ClickHouse Synchronization Issues with Prefix Indexes
- Fix segfault in topK aggregate function
- Do not restore WAL parts if in_memory_parts_enable_wal is disabled
- Fixed ClickHouse MySQL Connection Resumption Issue
- Fixed empty system.stack_trace table in daemon mode
- Fixed interactive mode behavior in clickhouse-client for multiline queries.
- Fixed rare server connection issue
- Fixed ALTER Query Hang Due to Killed Mutation on Different Replica
- Fixed Cache Size Estimation Bug in ClickHouse
- Fixed ORDER BY with optimize_redundant_functions_in_order_by setting
- Fixed Duplicate Rows Caused by Incorrect Optimization in DISTINCT Queries
- Fixed Crash with JOIN Table and LowCardinality Types
- Fixed Set Index Invalidation with Const Columns in Subquery
- Fixed ColumnConst Comparison Crash Issue
- Fixed ON CLUSTER Query Hang for Non-Leader ReplicatedMergeTreeTables
- Fixed bug in fuzzBits function related to issue #16980
- Avoid Network Errors for Cancellable Remote Queries
- Fixed Big Integer Casting from Double Error
- Reresolve IP for format_avro_schema_registry_url on errors
- Fixed Server Crash During ALTER TABLE Operations
- Blame Info Calculation Error in clickhouse-git-import
- Fixed Order By Optimization with Monotonous Functions
- Fix Optimization for Group By with Join Support
- Install Script: Ensure Subdirectories in Config Folders for Docker Builds
- Fix Illegal Argument Type Error for ORDER BY Queries
- Abort multipart upload for empty WriteBufferFromS3
- Fixed Crash with 'any' Function Argument Usage
- Fixed ClickHouse to return correct number of affected rows for INSERT queries via MySQL protocol
- Fixed Uncontrolled Growth of TDigest
- Fixed remote query failure with 'if' suffix in Aggregate function
Build/Testing/Packaging Improvement
29- Clang-11 Adopted for Production ClickHouse Build
- Now Using Clang-11 for ClickHouse CI Builds
- Switch binary builds to clang-11 for multiple platforms
- All test images updated to use llvm-symbolizer-11
- Allow Building with llvm-11
- Switch to clang-tidy-11
- Use LLVM's Experimental Pass Manager by Default
- Limit C++ Build Time to 10 Minutes and Memory Usage to 10 GB
- Make performance tests more stable by splitting test and profile runs
- Attempt to Enhance Performance Test Reliability Using Transparent Huge Pages
- Convert ClickHouse to Python 3
- Fail Early in Functional Tests for Server Response Failures
- Allow AArch64 ClickHouse Server to Run Without Configs
- Improvements in CI Docker Images: Removal of ZooKeeper and Streamlined Test Config Installation
- Fix CMake Options Forwarding in Fast Test Script
- Added Single-Command Hardware Benchmark Script
- Splitting of Large Test into Smaller Tests
- Maybe fix MSan report in base64 on AVX-512 servers
- Reformat and Cleanup Integration Test Code in Python Files
- Fix unstable test case for empty transaction in MaterializeMySQL
- Attempt to Speed Up Build
- Speed up build by removing unused headers
- Fix OSX Build Failure
- Enable ccache by default in cmake if found in OS
- Control CI Builds Configuration in ClickHouse Repository
- CMake File Updates: Option Descriptions, Defaults, and Removals
- Make Debug Binary Size Approximately 50 Mb Smaller
- Use std::filesystem::path in ConfigProcessor for File Path Concatenation
- Fix debug assertion in bitShiftLeft() for negative big integers
Improvement
34- Enable Atomic Database Engine by Default for New Databases
- Add Support for Specialized Codecs in Columns with Subtypes
- Dynamic Reload of Zookeeper Config
- Allow ALTER ... ON CLUSTER Queries Regardless of Internal Replication Setting
- Now joinGet Supports Multi-Key Lookup
- Wait for DROP/DETACH TABLE to Complete with NO DELAY or SYNC in Atomic Database
- Change Version Column Type in VersionedCollapsingMergeTree with ALTER Query
- Unfold macros in zookeeper_path during replicated table creation and restrict RENAME TABLE to prevent path breakage
- Function now accepts timezone argument, closing issue 15264
- Do not allow connections to ClickHouse until initialization scripts are executed
- Added optimize setting to EXPLAIN PLAN query for automatic query plan optimizations
- Proper Exception Message for Incorrect Number of Arguments in CAST
- Add option to disable TTL move on data part insert
- Ignore Key Constraints in Mutations with Index and Primary Key Flags
- Allow Replicated Table Drop After Failed Attempt Due to ZooKeeper Session Expiration
- Fixed settings constraint violation in SELECT with distributed tables
- Provide load_balancing_first_offset query setting for replica control
- Show Subqueries for SET and JOIN in EXPLAIN Result
- Allow Multi-Volume Storage Configuration in Distributed Storage
- Construct query_start_time and query_start_time_microseconds from timespec
- Support for Disabling Persistency in StorageJoin and StorageSet
- COLUMNS Function Updated to Apply Column Transformers
- Add merge_algorithm to system.merges table for better merging inspections
- Fix Memory Leak from Zookeeper Exists Watch
- Allow Parallel Execution of Distributed DDL in ClickHouse
- Add QueryMemoryLimitExceeded Event Counter
- Fix trailing whitespaces in query formatting
- ClickHouse Improvement: Addressing Non-Monotonic Binary Operators in Partition Expression Handling
- Add Option to Skip Access Checks for DiskS3 Experimental Feature
- Speed up server shutdown with ongoing S3 requests
- SYSTEM RELOAD CONFIG Exception Handling Update
- Support Semicolon as Data Terminator for Inline INSERTs in ClickHouse Client
- Support Custom Codecs in Compact Parts
- Workaround for S3 URL Path Issues with Nginx Proxy
New Feature
30- Background Data Recompression: Specify TTL RECOMPRESS Codec for MergeTree Tables
- Add parallel quorum inserts
- Settings for Enhanced Data Durability in Non-Replicated Setups
- When Writing Duplicate Blocks to Replicas, Ensure Local Writing Instead of Ignoring
- Support for Named Subqueries with WITH Syntax
- Introduce enable_global_with_statement to Propagate WITH Statements in Select Queries
- Secure Inter-Cluster Query Execution Enhancements
- Add Ability to Remove Column Properties and Table TTLs in ClickHouse
- Added RawBLOB Format for Unescaped Single Value Input/Output
- Add reinterpretAsUUID function for big-endian byte string conversion to UUID
- Implement force_data_skipping_indices Setting
- Add output_format_pretty_row_numbers setting for Pretty formats
- Added Query Obfuscation Tool for Improved Testing
- Add null() function for table structure in ClickHouse
- Added formatReadableQuantity function for human-friendly numeric output
- Add LineAsString format for parsing sequences of lines as single String fields
- Add JSONStrings format for string array output
- Add Support for Raw Column Format in Regexp
- Allow Configurable NULL Representation for TSV Output Format
- Support Decimal Data Type for MaterializeMySQL
- Add SHOW DATABASES LIKE 'xxx' Feature
- Added script to import git repository as sample dataset in ClickHouse
- Now insert statements support asterisk with column transformers
- New settings for query complexity limits in distributed queries
- Allow User-Defined Settings for ReplicatedMergeTree Storage in Config File
- Add mapPopulateSeries function
- Supporting MySQL decimal and datetime types in ClickHouse
- Introduce event_time_microseconds field to system log tables
- Add event_time_microseconds to system.asynchronous_metric_log and system.metric_log tables
- Add query_start_time_microseconds field to system.query_log and system.query_thread_log tables
Performance Improvement
11- Enable Default Compact Parts for Small Part Efficiency
- Improve quantileTDigest performance to fix issues #2668 and #15542
- Significant Memory Usage Reduction in AggregatingInOrderTransform
- Faster 256-bit Multiplication Improvement
- Improve performance of 256-bit types using uint64_t as base type for wide integers
- Explicit Use of Temporary Disk for Vertical Merge Data
- Use S3 DeleteObjects for Efficiency Instead of Multiple DeleteObject Calls
- Fix DateTime comparison using faster implementation
- Improve GROUP BY Performance for FixedString Type
- Only mlock Code Segment for clickhouse-server to Reduce Memory Usage
- ClickHouse Binary Size Reduction via Link Time Optimization