v.20.5
Backward Incompatible Changes
8- Return Zero for COUNT(DISTINCT) and uniq Functions When All Values Are NULL
- Added Check for Correct User-Level Settings Placement in Configuration
- Setting input_format_with_names_use_header Enabled by Default
- Remove experimental_use_processors Setting
- Update zstd to 1.4.4 with minor performance improvements and clarification on replica error messages
- Added Check for Meaningless Codecs and Control Setting
- Kafka Default Setting Changes in ClickHouse
- Preventing Part Intersects Errors During ClickHouse Version Upgrade
Bug Fix
159- Fix UBSan Report in Decimal Parse
- Fix floating point exception in DateTime64 parsing
- Fix rare crash with Nullable column in prewhere condition
- Don't Allow arrayJoin Inside Higher Order Functions to Maintain Protocol Synchronization
- Fix Comparison Issue Between FixedString and Constant String
- Fix NULL condition handling in if statements
- Fix excessive thread usage for queries
- Fixed Scalar Exception in SELECT with Scalar Subquery in Merge Tree Table
- Fix Unexpected Behavior in SELECT Queries
- Replicated Fetches Cancelled During Metadata Alter
- Parse Zookeeper Metadata Before Equality Check
- Fixed LOGICAL_ERROR in Values Input Format Due to Incorrect Type Deduction
- Fix ORDER BY ... WITH FILL for Const Columns
- Fix rare race condition in SYSTEM SYNC REPLICA command
- Pass Proper Timeouts for XDBC Bridge Communication
- Fix LIMIT n WITH TIES with ORDER BY aliases
- Fix Pipeline Stuck for Parallel FINAL Selects
- Fix mutation state error in system.mutations
- Fix CREATE USER query syntax highlighting
- Add Case-Insensitive Support for Regular Expressions
- Remove Count Query Optimization for Row-Level Security
- Fix Bloom Filters for String Data Skipping Indices
- Database Creation at Startup Requires -q Option
- Fix Block Structure Mismatch Error for Sampling Queries from Buffer Table
- Fix ClickHouse Client Exit Code for Specific Exception Case
- Fix Race Conditions in ReplicatedMergeTree CREATE/DROP Operations
- Fix log message error regarding cache size at server startup
- Fix offset size error in PREWHERE column subqueries with ARRAY JOIN
- Fixed Rare Segfault in SHOW CREATE TABLE
- All HTTP Session Queries Share a Fixed Query ID
- Clickhouse-server Docker Container Prefers IPv6 for Server Aliveness Checks
- Fix error with data compression when using direct IO and PREWHERE
- Fix shard_num and replica_num for <node> affecting use_compact_format_in_distributed_parts_names
- Fix async INSERT into Distributed with prefer_localhost_replica=0 and no internal_replication
- Fix Memory Leak in Aggregation with -State Functions
- Fix Pipeline Stuck Exception for INSERT SELECT FINAL with Multiple Streams
- Fix incorrect count results in multi-table queries
- Fix Compressed Size Return for Codecs
- Fix server crash with non-literal compression codec arguments
- Fix uninitialized memory read in MergeTree shutdown on table creation failure
- Fix Crash in JOIN over LowCardinality(T) and Nullable(T)
- Fix Error Code for Incorrect USING Key
- Fixed geohashesInBox for arguments outside latitude/longitude range
- Better error handling for joinGet() functions
- Fix Pipeline Stuck Error in Queries with External Sort and Limit
- Remove Redundant Lock in ReplicatedMergeTree Parts Send
- Fix \G support in clickhouse-client for multiline mode
- Fix segfault in Lazy database usage
- Fix crash in direct selects from Join table engine and correct nullability
- Fix crash in quantilesExactWeightedArray
- Merges Halted Before Metadata Changes in ALTER Queries
- Make MATERIALIZED VIEW Writing Parallel with parallel_view_processing = 1
- Fix visitParamExtractRaw for Unbalanced JSON Strings
- Fix rare race condition in ThreadPool
- Fix minor data race in clickhouse-copier identified by integration tests
- Fix Uninitialized Memory in `toIntervalSecond` Conversion
- Fix Index Analysis Issue with Array Column in Primary Key
- Fix Execution Speed Estimation Bug and Improve Usability in ClickHouse Client
- Fix Crash on Invalid SET DEFAULT ROLE Arguments
- Fix Crash When Reading Malformed Protobuf Data
- Fixed bug in cache dictionary returning default value for expired string fields
- Fix Block Structure Mismatch Error in QueryPipeline for VIEW with Inner Query Constants
- Fix Invalid Status Exception for Associated Output
- Primary Index Check Added to CREATE Query Validation
- Fix error capturing column in higher-order functions with Array of LowCardinality
- Fixed S3 Globbing Issue for Large Key Sets
- Fix index calculation for data skipping in merged columns in ClickHouse
- Fix intermitent hang during DROP of table engine=Kafka
- Fix Thread Reservation for Simple Queries Optimization
- Remove Logging from Mutation Finalization Task if No Finalization Occurs
- Fixed Deadlock During Server Startup After Update
- Fixed memory leak in registerDiskS3
- Fix "No such name in Block::erase()" Error with JOIN and PREWHERE
- Fix missed data issue during Kafka engine table termination
- Fixed parseDateTime64BestEffort Argument Resolution Bugs
- Enhancements to ALTER query: Support for ADD/DROP/RENAME on a single column and clearer error messages
- Fixed S3 URL Parsing
- Fix Memory Tracking for Two-Level GROUP BY with LIMIT
- Fix rare use-after-free error in MergeTree on failed table creation
- Fix Metadata and Data Handling for Atomic Database
- Fix server crash on concurrent ALTER and DROP DATABASE queries with Atomic engine
- Fix raw data size in getRawData() method
- Fix Two-Level Aggregation Incompatibility in ClickHouse 20.1 and Earlier
- Avoid Sending Partially Written Files in DistributedBlockOutputStream
- Fix Crash in SELECT count(notNullIn(NULL, []))
- Fix for Table Engine=Kafka DROP Hang Issue
- Execution of Multiple ALTER RENAME Commands in ClickHouse
- Fix Race Condition in Aggregate Function State Retrieval with Memory Engine
- Fix Backward Compatibility with Tuples in Distributed Tables
- Fix SIGSEGV in StringHashTable for non-existent keys
- Fixed WATCH hangs after LiveView table drop with Atomic engine
- Fixed ReplicatedMergeTree Bug Causing ALTER on OPTIMIZE to Hang
- Constraints Updated When Column Renamed
- Fix uninitialized memory read in cache dictionary
- Fix Column Order in Block::sortColumns() and Add Buffer Engine Test
- Fix ODBC Bridge Identifier Quoting Issue
- Fix UBSan and MSan Report in DateLUT
- Make Use of src_type for Correct Type Conversion in Key Conditions
- Get Rid of Old libunwind Patches to Improve clang Build Performance
- Fix avgWeighted for Floating-Point Weight Across Multiple Shards
- Fix parallel_view_processing to complete all insertions in MATERIALIZED VIEW despite exceptions
- Fix -OrNull and -OrDefault Combinator with -State
- Fix Crash in generateRandom with Nested Types
- Fix Data Corruption in LowCardinality(FixedString) Key Column for SummingMergeTree
- Fix primary key usage with 'FINAL' modifier and 'ORDER BY' optimization
- Fix buffer overflow in h3EdgeAngle function
- Fix Totals Disappearance Issue in Query Results
- Fix HTTP Insert Atomicity Issue #9666
- Fix redundant IN operator usages in query
- Fixed HTTP Request Bug on Client Close with Readonly Queries
- Fix Parameter Order in AggregateTransform Constructor
- Fix Remote Query Parallel Execution with Memory-Efficient Aggregation Enabled
- Fix Incorrect Row Count for Queries with LIMIT
- Fix bug causing locks on concurrent alters with many table parts
- Fix nullptr dereference in StorageBuffer on server shutdown before table startup
- Fix Optimization of Predicates in Distributed Queries with HAVING Clause
- Fix optimize_skip_unused_shards for LowCardinality
- Fix segfault in StorageBuffer on server startup
- On SYSTEM DROP DNS CACHE Query and Its Impact on Connection Caches
- Fixed Scalar Results in Inner Query for MATERIALIZED VIEW with Dependent Table
- Fixed Condition Variable Handling for Synchronous Mutations
- Fix possible crash in createDictionary() during loadStoredObject() execution
- Fix BloomFilter false positive error to accept double values between 0 and 1
- Fix SELECT Column ALIAS with Default Expression Type Mismatch
- Implemented DateTime64 and String Value Comparison
- Fix index corruption after merging compact parts
- Disable Default GROUP BY Sharding Key Optimization and Fix WITH ROLLUP/CUBE/TOTALS
- Fix issues with INSERT and distributed sends in ClickHouse
- Fixed max_rows_to_sort limit error
- Get Dictionary and Check Access Rights Once per Function Call
- Fixed Index Analysis in MergeTree Tables Leading to Incorrect Pruning
- Fixed Thread Limiting for Local Replica Selects
- Fixed Performance Issues with Large Tuples in IN Clause
- Fixed Memory Tracking for Input Format Parallel Parsing
- Fixed Bloom Filter Index with Constant Expression
- Fixed SIGSEGV in StorageKafka when broker is unavailable
- Added support for if function with Array(UUID) arguments
- Fixed Aliases for Function Any
- Fixed race condition in external dictionaries leading to server crash
- Remove Distributed Table Data on DROP TABLE
- Fixed Bug Causing Broken Old Parts After ALTER DELETE Query with Mixed Granularity Parts Enabled
- Better exception for function in with invalid argument count
- Fixed Race Condition Causing Data Duplication in Live View Tables
- Fixed Performance Issue in Compact Parts Reading
- Fixed Backwards Compatibility in AggregateFunction(avg) Binary Format
- Fixed deadlock with enabled text_log
- Fixed Overflow for Large LIMIT or OFFSET Values in ClickHouse
- Fixed Segfault Issue in StorageMerge
- Revert change from #11079 to resolve #12098
- Avoid Exception with Negative or Floating Point Constants in Indexed Tables WHERE Clause
- Allow Clearing Column with Dependent Default Expressions Fixes #12333
- Fixed Aggregate Functions in TOTALS/ROLLUP/CUBE for -State and Nullable Arguments
- Fixed SIGSEGV in Kafka Engine for batch errors
- Fixed SummingMergeTree Behavior with Partition Key Columns
- Fixed Query Transformation for External DBMS with Aliases
- Fixed ZooKeeper Metadata Bug for ReplicatedVersionedCollapsingMergeTree Tables
- Fixed Thread Limiting for VIEW Selections
- Fixed Crash in JOIN with LowCardinality Type Using Partial Merge Algorithm
- Fixed NULL condition handling in if() function
Build/Testing/Packaging Improvement
67- Enable clang-tidy for programs and utils
- Remove tzdata Dependency in ClickHouse Timezone Handling
- Added MSan and UBSan Stress Tests
- Print Compiler Build ID in Crash Messages
- Added Test for Mutation Functionality After FREEZE Query
- Don't Allow "fail" Substring in Test Names to Improve Search Convenience
- Removes unused imports from HTTPHandlerFactory
- Added random sampling to copier execution to prevent simultaneous query errors and adjusted timeout settings
- Fix Missed Include in ClickHouse
- Speed up build by removing old examples and fixing orphan tests
- Increase ccache Size for CI Builds
- Leave only unit_tests_dbms in deb build
- Update librdkafka to version 1.4.2
- Refactor CMake Build Files #11390
- Fix flaky integration tests
- Add UBSan Support for Unit Tests
- Remove Redundant Timeout from Integration Test test_insertion_sync_fails_with_timeout
- Better Check for Hung Queries in ClickHouse Test
- Emit Warning for Debug Builds and Sanitizers
- Clickhouse-test Adds Server Aliveness Check Before Tests
- Fix Flaky Test 00731_long_merge_tree_select_opened_files.sh Due to Potential Race Condition
- Repeat CI Test on curl Timeout to Address Issue #11267
- Add test for Join table engine, resolves issue #9158
- Fix Non-Significant Errors in Unit Tests
- Linker Command Adjustment for cctz Library Integration
- Split server into program and library
- Improve build scripts for protobuf and gRPC
- Enable Non-Functional Performance Test
- Create Root S3 Bucket for Tests Before CH Instance Startup
- Add performance test for non-constant polygons
- Fix 00979_live_view_watch_continuous_aggregates Test
- Add tmpfs support for Zookeeper in integration tests
- Wait for odbc-bridge with exponential backoff in CI environment
- Fix non-deterministic test (#10989)
- Added test for empty external data
- Database Recreation Enhances Test Isolation
- Added More Asserts in Columns Code
- Better Sanitizer Cooperation with Query ID in Failure Messages
- Fix race condition in Split build smoke test check
- Fix False MSan Report in MergeTreeIndexFullText
- Add MSan Suppression for MariaDB Client Library
- GRPC build fix: Updated makefile to include correct protobuf file path
- Enable Extra Warnings for Base, Utils, Programs
- Suppressions of library warnings mistakenly declared as public in ClickHouse
- Restore Accidentally Deleted Patch in #10396
- Fix performance tests errors, part 2
- Fix Performance Test Errors
- Update cross-builds to use clang-10 compiler
- Update RPM Package Installation Instructions
- Fixing test for insert_into_watch_live_view.py
- Fix and Re-enable 00979_live_view_watch_continuous_aggregates.py Test
- Fix OOM in ASan stress test
- Fix UBSan Report in HashTable After Clang-10 Migration
- Remove External Call to ld Linker During tzdata Processing in Compile Time
- Allow lld to Link Blobs in ClickHouse
- Fix UBSan Report in LZ4 Library
- Update LZ4 to Latest Dev Branch
- Added Auto-Generated Machine-Readable File for Stable Versions
- Fix capnproto version check for UnalignedFlatArrayMessageReader
- Lower Memory Usage in Tests
- Fix hard coded timeouts in live view tests
- Increasing Timeout for Client in Tests
- Enable ThinLTO for Clang Builds
- Adding Fuzzers and Preparing for OSS-Fuzz Integration
- Fix FreeBSD Build Issue #10150 by Ivan
- Add pytest framework for query tests
- Install ca-certificates Before apt-get Update in Dockerfile
Experimental Feature
3Improvement
54- Apply TTL for Old Data After ALTER MODIFY Query in ClickHouse
- Enhanced Parsing of C-style Backslash Escapes for Better LIKE and Regular Expression Compatibility in ClickHouse
- When reading Decimal values, truncate extra digits for MySQL and PostgreSQL compatibility
- Allow Dropping and Renaming of Replicated Tables with ZooKeeper Metadata Issues
- Slightly Improve Decimal Reading Diagnostics
- Fix sleep duration in signal handler functionality
- Linux OS Performance Metrics Functionality Without CAP_NET_ADMIN
- Added hostname Alias to hostName Function
- Added Distributed DDL Support for Cross Replication Clusters
- Emit Warning for Unavailable Listen Addresses at Server Startup
- Default User and Database Creation in Docker Image
- Fix Multiline Query Logging for String Literals, Identifiers, and Comments
- Multiple Names Allowed in User and Role Commands
- Add Distributed DDL Support for Cross-Replication Clusters
- Clear Password from Command Line in ClickHouse Client and Benchmark
- Don't Use Mismatched ELF Debug Info to Avoid Incorrect Stack Traces
- Return NULL/zero for incomplete parsing in parseDateTimeBestEffortOrNull/Zero functions
- Skip Empty URL Parameters in Requests
- Allow Use of groupArrayArray and groupUniqArrayArray as SimpleAggregateFunction
- Allow Implicit String Comparisons for Index Condition Analysis
- Support for Default HTTP Handlers in ClickHouse
- Improve Kafka engine input formats and resolve performance issues
- Improve multiple joins rewriter logic to fix unknown columns error for lambda aliases
- Better Exception Message for Columns Declaration Parsing Error
- Improve enable_optimize_predicate_expression=1 Logic for VIEW
- Adding PREWHERE Support for Live View Tables
- Automatically Update DNS Cache for User Connection Validation
- OPTIMIZE FINAL Forces Merge Despite Concurrent Operations
- Suppress Output of Cancelled Queries in ClickHouse Client
- History File Updated Per Query to Prevent Race Conditions in Multi-Client Use
- Better log messages during configuration reload
- Remove Trailing Whitespaces from Queries in ClickHouse Client
- Add Output Format Pretty Max Value Width Setting
- Better Exception Message for Memory Mapping Shortages
- Support for (U)Int8, (U)Int16, and Date in ASOF JOIN
- Support kafka_client_id Parameter for Kafka Tables in ClickHouse
- Keep DistributedFilesToInsert Metric Value on Exceptions
- Add Support for Multi-Word Data Type Names for SQL Compatibility
- Provide Synonyms for Data Types
- Query Log Enabled by Default
- Show Authentication Type in Users Table and SHOW CREATE USER Query
- Remove DROP DATABASE Data for Memory Engine
- Set Thread Names and Enable rdkafka Logs in ClickHouse
- Support for Unicode Whitespace in Queries
- Allow large UInt types as index in tupleElement function
- Respect prefer_localhost_replica/load_balancing on INSERT into Distributed
- Introduce min_insert_block_size settings for Materialized Views
- Fix Exception from Replicated Queue During Server Shutdown
- Fix negative results in variance calculations for varSamp and varPop
- Better DNS Exception Message Fixes #10813 and #10828
- Change HTTP Response Code to 400 Bad Request for Parse Errors
- Print Message for Clickhouse-Client Version Check
- Adding Support for INSERT INTO [db.]table WATCH Query
- Allow passing quota_key in clickhouse-client
New Feature
48- TTL DELETE WHERE and TTL GROUP BY for Automatic Data Coarsening in Tables
- Implementation of PostgreSQL Wire Protocol
- Added System Tables and SHOW Commands for Users, Roles, and Settings in ClickHouse
- Support for ODBC Table Function in ClickHouse
- Add Linux perf_events Query Performance Metrics in ClickHouse
- Support NULL and NOT NULL Modifiers in CREATE Query
- Add ArrowStream Input and Output Format
- Support Cassandra as External Dictionary Source
- Added direct data loading layout in ClickHouse
- Added complex_key_direct layout to dictionaries for non-local query execution
- Added MySQL Style Global Variables Support for Protocol Compatibility
- Added Syntax Highlighting to clickhouse-client Using replxx
- minMap and maxMap Functions Added
- Add system.asynchronous_metric_log table for historical metrics logging
- Add extractAllGroupsHorizontal and extractAllGroupsVertical functions
- Add SHOW CLUSTER(S) Queries
- Add netloc function for extracting network location in Python style
- Add Virtual Columns for Kafka to Access Message Headers
- Add _timestamp_ms Virtual Column for Kafka Engine
- Add randomFixedString function
- Add fuzzBits function to randomly flip bits in a string
- Allow Comparison of Numbers with Constant Strings in ClickHouse
- Add round_robin load_balancing mode
- Add cast_keep_nullable setting for nullable type casting
- Added Column Position to system.columns and system.parts_columns Tables
- ON CLUSTER Support for SYSTEM FLUSH DISTRIBUTED and STOP/START DISTRIBUTED SEND
- Add system.distribution_queue table
- Support for Kafka Format Settings and Table-Level Configuration in ClickHouse
- Add port function to extract port from URL
- Now dictGet* functions accept table names
- clickhouse-format tool enhances multi-query formatting with -n argument
- Possibility to Configure Proxy-Resolver for DiskS3
- Make pointInPolygon Compatible with Non-Constant Polygon Arrays
- Added move_ttl_info to system.parts for move TTL introspection
- Possibility to work with S3 through proxies
- Add NCHAR and NVARCHAR Synonyms for Data Types
- Resolved #7224: Added FailedQuery, FailedSelectQuery, and FailedInsertQuery Metrics to system.events Table
- Add jemalloc statistics to system.asynchronous_metrics
- Allow Default S3 Credentials and Custom Auth Headers
- Added DateTime64 Import/Export Functions with Precision Options
- Allow Specifying MongoDB URI for Dictionaries
- OFFSET keyword now usable without LIMIT clause
- Added system.licenses table for third-party library licenses in contrib directory
- New function toStartOfSecond for DateTime64 in ClickHouse
- Add JSONAsString Input Format for Multiple JSON Objects
- Improved Memory Profiling with Finer Granularity and Sampling Profiler
- SimpleAggregateFunction Now Supports sumMap
- Support ALTER RENAME COLUMN for Distributed Table Engine
Performance Improvement
22- Allow Concurrent Operations for Multiple Replicas
- Optimization of GROUP BY with Table Sorting Key Using optimize_aggregation_in_order Setting
- Final Selects Execution in Parallel with max_final_threads Setting
- Improve INSERT Query Performance and Fix Constraint Issues in ClickHouse
- Optimization for Removing Aggregators in GROUP BY Keys using optimize_aggregators_of_group_by_keys Setting
- New Optimization for Moving Operations Out of Any Function
- Improve clickhouse-client performance in interactive mode with Pretty formats
- Improved Query Performance for ORDER BY with Small LIMIT in ClickHouse
- Add Runtime CPU Detection and Multi-Target Code Generation
- Enable mlock for ClickHouse Binary by Default to Prevent Paging
- Make Queries Faster with SUM Function and No GROUP BY
- Improving Radix Sort Efficiency by Reducing Redundant Data Moves
- Sort Left Table Blocks in MergeJoin with Memory Buffering and New Setting
- Remove Duplicate ORDER BY and DISTINCT from Subqueries for Optimization
- Feature to Optimize GROUP BY Function Keys in ClickHouse
- New Optimization for Aggregate Functions in ClickHouse
- Use Poco HTTP Client for S3 to Enhance Performance and Reduce Memory Usage
- Fix Kafka Performance Issue with Reschedules Based on Limits
- Enable percpu_arena for jemalloc to Reduce Memory Fragmentation
- Optimize Memory Usage in S3 HTTP Client Response Handling
- Adjust Kafka Default Settings for Improved Performance
- Index Not Used for IN Operator with Literals, Fixes Performance Regression #10574