v.22.9
Backward Incompatible Changes
5- Upgrade Procedure for ReplicatedMergeTree Tables from ClickHouse 20.3 to 22.9
- Remove Unused and Undocumented Functions `accurate_Cast` and `accurate_CastOrNull`
- Add Documentation Test and Rename MeiliSearch Table Function to meilisearch
- Add test for function documentation; fix case sensitivity for lemmatize, synonyms, and stem functions
- Make YAML Config Interpretation More Conventional
Bug Fix
82- Fix potential data loss due to AWS SDK bug when using ClickHouse with S3
- Malicious Native Format Data May Cause Crash
- Fix null pointer issue in categorialInformationValue aggregate function
- Writing Data in Apache ORC Format May Cause Buffer Overrun
- Fix Memory Safety Issues in `encrypt` and `contingency` Functions
- Fix MergeJoin Bugs When Not Processed Is Not Null
- Fix Decimal Precision Loss in IN Operator
- Fix Nested Column Filling for Multiple Levels
- Fix SYSTEM UNFREEZE Query for Ordinary Database
- Fix Unused Unknown Columns from WITH Statement
- Fix ORDER BY Query Analysis with Window Functions
- Fixed Unknown Identifier Exception for WINDOW ORDER BY/PARTITION BY Expressions
- Limit Analyze Depth for Queries to Prevent Excessive Analysis Time
- Fix rare bug with column TTL in MergeTree engines
- Use DNS Entries for IPv4 and IPv6 if Available
- Allow Reading Snappy Compressed Files from Hadoop
- Fix crash parsing variadic arrays in experimental Object feature
- Fix input_format_tsv_skip_first_lines in settings
- Fix Race Condition in MaterializedPostgreSQL Startup and Relcache Callback List Slot Error
- Fix Decimal Math Overflow Error in DateTime64 Parsing
- Fix vertical merge issue with deleted lightweight rows
- Fix segmentation fault in URL table engine with compression enabled
- Fix logical error in arrayElement function with Map in ClickHouse
- Fix race condition in filesystem cache
- Removed Skipping of Mutations in Unaffected MergeTree Partitions
- ClickHouse Server Crash Due to Occupied gRPC Port in Runtime Configuration
- Fix base58Encode/base58Decode for leading 0/'1' handling
- keeper-fix: resolve log access race during snapshot installation
- Fix toFixedString Function Short Circuit Execution
- Fix SQLite int8 to int64 column conversion in ClickHouse
- Fix stack overflow in recursive Buffer tables
- Fix memory limit issue during ProcessList query insertion
- Fix LOGICAL_ERROR with max_read_buffer_size=0 during reading marks
- Fix memory leak in MVs without query context
- Fix CSV Schema Inference Error on EOF
- Fix logical error in write-through cache related to file segment completion
- Make GROUPING Function Consistent with SQL and Other DBMS
- Fix for host_regexp functionality in /etc/hosts resolution
- Fix Incremental Backups for Log Family
- Fix rare data loss bug in zero-copy replication
- Fix key condition crash from identical set expressions in different columns
- Fix nested JSON Objects schema inference
- Fix Deletion of Empty 3-Digit Prefix Directory for Filesystem Cache Files
- Fix uncaught DNS_ERROR on failed connection to replicas
- Fix bug in subquery column removal
- Fix Memory Allocation for Remote Read Buffers
- Fixed issue allowing users with revoked grants to drop databases
- Fix path comparison in ClickHouse Keeper write requests
- Fix deadlock in WriteBufferFromS3
- Fix Access Rights for DESCRIBE TABLE Functions
- Remove Incorrect Parser Logic for WITH GROUPING SETS to Prevent Nullptr Dereference
- Fix Possible Segfault During ClickHouse Keeper Shutdown
- Fix segfaults and memory issues in aggregate function combinators
- Fix query_views_log with Window views
- Disables optimize_monotonous_functions_in_order_by by default
- Fixed "possible deadlock avoided" error in database engine conversion
- Fix SIGSEGV in SortedBlocksWriter for Empty Blocks with Specific Settings
- Fix query result with trivial count optimization and array join
- Fix stack-use-after-return in GetPriorityForLoadBalancing::getPriorityFunc
- Fix Positional Argument Exception, Closes #40634
- Fix background cleanup of detached parts
- Fix Exponential Query Rewrite for Cross Joins with Where Clauses
- Fix logical error in write-through cache related to exception handling
- Fix String Log Entry in Filesystem Cache Log
- Fix for Incorrect Results in Queries with OFFSET and WHERE Clauses
- Fix Incorrect Query Results with query_plan_optimize_primary_key Enabled
- Do not allow invalid sequences to influence other rows in UTF8 settings
- Fix ALTER table queries for Object type columns
- Fixed "No node" Error in distributed_ddl_queue when Configuration is Missing
- Fix logical error in disk object storage regarding expected relative path
- Add Column Type Check for UUID Insertion in MsgPack Format
- Fix crash from async insert of malformed Object data
- Fix deadlock issue with async_socket_for_remote and parallel KILL
- Disables optimize_rewrite_sum_if_to_count_if by default
- Fix for ON CLUSTER Clause Handling in Replicated Databases
- Fix Deadlock Issues on Query Cancellation
- Fix Server Crash Issue with JBOD Feature
- Fix nullable fixed string conversion issue
- Prevent Crash with Incorrect Aggregation States in groupBitmap*
- Fix for Incorrect Results in Queries with ORDER BY and Limited Rows
- Fix read bytes/rows in X-ClickHouse-Summary with materialized views
- Fix Pipeline Stuck Exception for OFFSET Queries with Always False Condition in WHERE
Build/Testing/Packaging Improvement
23- Enforce Documentation for Every Setting
- Enforce Documentation for Current Metrics
- Enforce Documentation for Missing Profile Event Counters
- Allow minimal clickhouse-local build by correcting dependencies
- Calculate SQL Function Coverage in Tests
- Enforce Documentation for MergeTree Settings
- Prototype for Embedded Reference Documentation of Server Components
- Review of Queries from Updated Performance Tests
- Fix TGZ Packages #40681 by Mikhail f. Shiryaev
- Fix Debug Symbols in ClickHouse
- Extended CI Configuration for x86 SSE2-Only Build
- Switch to llvm/clang 15
- Continuation of #40938: Fix ODR Violation for Loggers Class and Related Issues
- Add macOS Binaries to GitHub Release Assets to Fix Issue #37718
- c-ares Library Integrated into ClickHouse Build System
- Remove dlopen from Main ClickHouse Code
- Disallow dlopen in ClickHouse Main Binary Due to Security Risks
- Add source field to deb packages and update nfpm
- Support for DWARF-5 in In-House DWARF Parser
- Add Fault Injection in ZooKeeper Client for Testing
- Add Stateless Tests with S3 Storage and Debugging
- Trying stress on top of S3 #36837
- Enable concurrency-mt-unsafe in clang-tidy
Experimental Feature
3Improvement
48- ReplicatedMergeTree Tables Read-Only Until ZooKeeper Connection Established
- Add enable_extended_results_for_datetime_functions for Date32 Results in ClickHouse
- CatBoost Model Evaluation Migration to clickhouse-library-bridge
- Add More Metrics for On-Disk Temporary Data
- Add warning_supress_regexp config option
- Add setting to disable limit on kafka_num_consumers
- Support SETTINGS in DELETE query
- Detailed S3 Profile Events per API Call Split for S3 ObjectStorage
- Two New Metrics in system.asynchronous_metrics: NumberOfDetachedParts and NumberOfDetachedByUserParts
- Allow CONSTRAINTs for ODBC and JDBC tables
- Limit Duplicate Printing of SETTINGS During Query Formatting
- Improve OpenTelemetry Context Propagation Across Threads
- ClickHouse Keeper: Conditional Listeners for interserver_listen_host
- Improve Recovery of Replicated User Access Storage After Errors
- Add TTL Support in EmbeddedRocksDB
- Add Schema Inference to ClickHouse Obfuscator
- Improve Arrow Format Dictionaries
- More natural conversion of Date types to narrower types in ClickHouse
- Fix Merge Table Index Issue Over View
- Custom Key Names for JSON Server Logs
- Custom Error Code for `throwIf` Function Implemented
- Improve Schema Inference Cache to Respect Format Settings
- Allow Date Parsing as DateTime and DateTime64
- Allow String to Date and Date32 Conversion for DateTime Formats
- Better support for nested data structures in Parquet format
- Support Reading Array(Record) into Flatten Nested Table in Avro
- Add read-only support for EmbeddedRocksDB
- Validate Compression Method Parameter for URL Table Engine
- Better URL Format Detection for Table Function with Query String
- Disable Projection in Grouping Set to Fix Incorrect Results
- Fix APPLY Column Transformer Format to Prevent Metadata Issues
- Support %z Descriptor for Timezone Offset in formatDateTime
- Interactive mode in clickhouse-client now supports '.' and '/' to rerun last command
- Fix MySQL Timeouts for Database Engine and Table Function
- Create Status File for Filesystem Cache Directory to Prevent Sharing Across Servers
- Add DELETE and UPDATE support for EmbeddedRocksDB storage
- ClickHouse Keeper: Shutdown Fix and Request Size Increase
- Fix race condition in WriteBufferFromS3 with TSA annotations
- Grouping Sets with group_by_use_nulls Affects Key Column Nullability
- Enhance INSERT Observability on Distributed Table
- More Low-Level Metrics for S3 Interaction
- Support Relative Path in Location Header After HTTP Redirect
- Apply HTTP Handler Changes On-the-Fly Without Restart
- ClickHouse Keeper: Close Active Sessions on Shutdown to Reduce Read-Only Errors
- Add Automatic SQL Commenting in ClickHouse Client
- Fix cache incompatibility when toggling do_no_evict_index_and_mark_files setting
- Add Configuration to Limit FixedString Column Size to 256
- Add has_lightweight_delete to system.parts
New Feature
9- Support auto-insert quorum for majority number
- Add Embedded Dashboards to ClickHouse Server for 90% Results with 1% Effort
- Added changeable_in_readonly settings constraint
- Add INTERSECT DISTINCT and EXCEPT DISTINCT Support
- Add JSONObjectEachRow Format and New JSON Settings in Import/Export
- Query Parameters in DESCRIBE TABLE Query Support
- Add Parquet Time32/64 Support by Converting to DateTime64
- Implement Apache Datasketches Set Operations
- Allow Error Recording to File for CSV and TSV Formats
Performance Improvement
18- Implement Compression of Marks and Primary Key
- Allow Asynchronous Loading of Marks with Threadpool Option
- Virtual Filesystem Over S3 with Random Object Names for Improved AWS Performance
- Account max_block_size for multi-threaded single-level aggregation.
- Software Prefetching in Hash Table Aggregation Enabled by Default
- Better support for optimize_read_in_order with constant sorting key columns in ClickHouse
- Filter Joined Streams for Full Sorting Join
- LZ4 Decompression Optimization by Skipping Empty Literals
- Speedup Backup Process with Native Copy in ClickHouse
- Do not obtain storage snapshot for each INSERT block to improve performance
- Implement Batch Processing for Aggregate Functions with Multiple Nullable Arguments
- Speed up reading UniquesHashSet from disk
- Fixed High Memory Usage in Table Mutations with Many Columns
- Enable vectorscan library on ARM for faster regexp evaluation
- Upgrade vectorscan to 5.4.8 for Enhanced Regexp Evaluation Performance
- Fix VFS Cache Fallback Issue at High Concurrency Levels
- Row Policy Filter Optimization for Immediate Empty Result
- Parallel Hash JOIN Optimization for Float Data Types