v.23.2
Backward Incompatible Changes
7- Extend toDayOfWeek() Function with Mode Argument for Week Start and Counting Options
- Rename max_query_cache_size to filesystem_cache_max_download_size
- Default user loses SHOW NAMED COLLECTION access permissions
- Settings Clause Application in Formatting Rules
- Remove Support for materialized_postgresql_allow_automatic_update
- Slight performance improvement for countDigits function and correction of countDigits(0) return value
- Disallow Certain Codec Combinations for New Columns in ClickHouse
Bug Fix
43- Flush Data Management in StorageRabbitMQ
- Use PODArray in sparkBar function for memory control
- Fix unsorted array return in quantilesExactExclusive and quantilesExactInclusive functions
- Fix uncaught exception in HTTPHandler with OpenTelemetry enabled
- Don't Infer Dates from 8-Digit Numbers to Avoid Data Errors
- Fix ODBC Bridge Connection Pooling Setting
- When Cache Callback is Called, Use Value Capture for Safety
- Fix Data Corruption with Combined Delta/DoubleDelta and Gorilla Codecs
- Correct Type Checking in N-gram Bloom Filter Index
- Report and Fix for Segfaults in c-ares Integration
- Fix Duplicate Primary Key Description in Projections
- Set Compression Method and Level for Backup
- Should Use select_query_typed.limitByOffset Instead of select_query_typed.limitOffset
- When using experimental analyzer, offset causes incorrect results in queries
- Backward Compatibility: Implicit Narrowing Conversion from UInt64 to IPv4 Required for INSERT Expression
- Bugfix for IPv6 Parser Handling Mixed IPv4 Addresses
- Add query_kind column to system.processes table and SHOW PROCESSLIST; fix max_concurrent_select_queries bug with INTERSECT/EXCEPT
- Fix crash in stochasticLinearRegression function found by WingFuzz
- Fix crash in SELECT queries with INTERSECT and EXCEPT for tables with sparse columns
- Fix DESC Sorting Read Optimization with FINAL
- Fix Nested Column Reading in Compact Parts
- Fix elapsed column error in system.processes
- Follow-up Fix for Replace Domain IP Types (IPv4, IPv6)
- Fix Environment Variable Substitution in Configuration
- Fix Incorrect Predicate Push Down with Grouping Sets
- Fix pipeline stuck error on fulls_sorting_join with constant keys
- Avoid Rewriting Tuple Functions as Literals During Formatting
- Fix Out of Bounds Error in LowCardinality(Nullable) Reading for Arrow Format
- Fix SYSTEM UNFREEZE Queries Failing with CANNOT_PARSE_INPUT_ASSERTION_FAILED Exception
- Fix Integer Overflow Crash in Aggregating Function Deserialization
- Fix LOGICAL_ERROR in Asynchronous Inserts with Invalid VALUES Data
- Fixed LOGICAL_ERROR in ALTER ... MOVE PART ... TO TABLE Query
- Fix s3Cluster Schema Inference for Parallel Distributed Insert Select
- Fix Nested Column Update Queries in ALTER TABLE Statements
- Scheduler Task Failure Handling in Multipart Upload
- Fix PREWHERE Handling in Merge for Different Default Column Types
- Fix crash with constant values in ORDER BY
- Do not throw exception for disk override on query level in config merge tree settings
- Fix Bug in LowCardinality Argument Processing in arrayMap Function
- Fixes issues #46557 and PR #46611 by Alexander Gololobov
- Fix Endless Restarts of ClickHouse-Server Systemd Unit by Disabling Timeout Logic
- Allocated memory buffers during asynchronous inserts led to false positive OOM exceptions
- Updated handling of on_expression in table_join for future analyze runs
Build/Testing/Packaging Improvement
10- Introduce GWP-ASan in LLVM Runtime to Address Issue #27039
- Add Randomization to Merge Tree Settings in Tests for Flakiness
- Enable HDFS Support in PowerPC and Fix Functional Tests
- Add systemd.service file for clickhouse-keeper, fixes issue #44293
- ClickHouse Moves Poco Fork to Base Directory
- Add Child Process Restart Option for ClickHouse Watchdog
- ClickHouse Docker Restart Configuration Update
- Fix Systemd Service File (#46461)
- Raised Minimum Clang Version for ClickHouse from 12 to 15
- Upgrade Intel QPL to v1.0.0 and link libaccel-config statically
New Feature
18- Add StorageIceberg and iceberg function for S3 access
- Allow Configuring Storage with New Disk Settings in ClickHouse
- Expose ProfileEvents Counters in system.part_log
- Enrichment of ReplacingMergeTree Engine for Duplicate Insertion Handling
- Add generateULID function, closes #36536
- Add corrMatrix, covarSampMatrix, and covarPopMatrix Aggregate Functions
- Introduce arrayShuffle Function for Random Array Permutations
- Support Fixed-Size Binary and Fixed-Length Byte Array Types in Arrow and Parquet
- Add last_exception_time column to system.replication_queue
- Add user-defined keys/seeds support for SipHash128
- Allow Three-Argument Version for Table Function Format
- Add JodaTime format support for x, w, S
- Support for NTILE Window Function
- Add 'final' setting to apply FINAL modifier to all tables
- Added arrayPartialSort and arrayPartialReverseSort Functions
- New HTTP Parameter for Client Protocol Version in Native Format
- Add regexpExtract Function for Compatibility with Spark's REGEXP_EXTRACT
- Add JSONArrayLength Function to Return JSON Array Element Count
Improvement
57- Enable Default Retries for INSERT on ZooKeeper Session Loss
- Add Ignore Unknown Keys in JSON for Named Tuples
- Support Optimizing WHERE Clause with Sorting Key Movement to PREWHERE for FINAL Queries
- Add new metrics for backups: num_processed_files and processed_files_size
- Added Retries for Interserver DNS Errors
- Keeper Enhancement: Preallocate Disk Space and Introduce max_log_file_size Setting
- Optimize Replica Delay API for Read-Only Instances
- Ask for Password in ClickHouse-Client When Empty Password is Wrong
- Mark Gorilla Compression on Non-Float Columns as Suspicious
- Show Replica Name in Postpone Reason Column During Merge
- Save exception stack trace in part_log
- regexp_tree dictionary updated for compatibility with uap-core
- Updated SYSTEM SYNC REPLICA Checking, Resolving Issues #45508 and #45648
- Rename replication_alter_partitions_sync setting to alter_sync
- Support for LowCardinality Data Types in generateRandom Table Function
- Experimental Query Result Cache Enhances Modular Configuration Settings
- Renaming "query result cache" to "query cache"
- add SYSTEM SYNC FILE CACHE command to execute sync syscall
- Add optional S3 setting allow_head_object_request to use GetObjectAttributes by default.
- Add Connection Settings Override Feature for ClickHouse Client
- Arrow format: support duration type and related PRs
- Extend Query Cache Logging for Better Caching Investigations
- Query Cache Server-Level Settings Now Reconfigurable at Runtime
- Hide Password in Logs for Named Collection Arguments in Table Functions
- Improve S3 Client Region and Redirection Handling
- Add Map, IPv4, and IPv6 support in generateRandom for testing
- Support Empty/NotEmpty for IP Types
- Column Splits for File Processing in ClickHouse
- Add MySQL Compatibility for SHOW ENGINES Query Support
- Improved Obfuscator Query Handling
- Improve Date Conversion for Boundary Value 65535
- Add `check_referential_table_dependencies` Setting for `DROP TABLE` Referential Checks
- Fix tupleElement to return Null for Null argument
- Throw Error for No S3 Wildcard Files Found
- Use Cluster State Data for Concurrent Backup/Restore Check
- ClickHouse Client Enhances Fuzzy Search with Exact Matching Algorithm
- Forbid Incorrect CREATE VIEW Syntax
- Storage Log Family Support for Setting Storage Policy
- Improve JSONColumns Format for Empty Results
- Add SipHash128 Reference Implementation
- Add metric for allocation times and bytes using mmap
- Function Length Argument Restriction in ClickHouse Functions
- Fix NaN Handling in Welch and Student T-Test Functions
- More Convenient Usage of Big Integers and ORDER BY WITH FILL in ClickHouse
- Add columns to system.tables for parts, active_parts, and total_marks
- Functions multi[Fuzzy]Match Reject Slow Regexes
- Insert Null as Default Behavior and Fix for LowCardinality Columns
- Prefer Explicit Access Keys for S3 Clients with Environment Credentials
- Add DATE_FORMAT() alias for formatDateTime() to enhance MySQL compatibility
- Add ProfileEvents and CurrentMetrics for Parallel Replicas Callback Tasks
- Add DELETE and UPDATE support for KeeperMap storage engine
- Allow RENAME Queries with Query Parameters
- Fix Parameterized SELECT Queries with REPLACE Transformer
- Exclude Internal Database from Asynchronous Metric Calculation
- Added last_exception_time column to distribution_queue table
- Support for IN Clause with Parameters in Parameterized Views
- Do Not Load Named Collections on Server Startup
Performance Improvement
29- Introduced Logic for Efficient PREWHERE Condition Handling in Future Releases
- Option to Aggregate Partitions Independently with Compatibility Check
- Allow Vertical Merge Algorithm with Compact Format for Memory Efficiency
- Optimize Parquet Reader with Batch Processing
- Add io_uring Method to local_filesystem_read_method for Improved Read Performance
- Rewrite Aggregate Functions Using if Expressions for Performance Improvement
- Improve lower/upper function performance with AVX512 instructions
- Remove Core Limitation for ClickHouse on High-Core Systems with SMT Disabled
- Improve multiIf function performance by 2.3x through columnar execution
- Add Fast Path for Empty Needle in Position Function
- Enable query_plan_remove_redundant_sorting Optimization by Default
- Increased HTTP Transfer Encoding Chunk Size for Enhanced Query Performance
- Fixed Performance of Short SELECT Queries with Large Array/Map/Nested Columns
- Improve Performance of Filtering for Big Integers and Decimal Types
- Performance Improvement by Reducing Overhead in ColumnNullable Filter Processing
- Make _part and _partition_id virtual columns LowCardinality(String) type
- Improve Decimal Conversion Performance When Scale Unchanged
- Allow increased prefetching for read data
- Rewrite arrayExists for Improved Performance in ClickHouse
- Fix Memory Usage for Vertical Merges and Stream Limits on Remote Disk
- Update zstd to v1.5.4 with Performance Improvements
- Fix Performance Degradation Related to Issue #39737
- replicas_status handle efficiently manages large replication queues
- Add AVX512 Support for Sum, Unary Arithmetic, and Comparison Functions
- Rewrote code for marks distribution to enhance performance, resolving issues #34527 and #43772.
- Remove Redundant DISTINCT Clauses in Query Optimization
- Query Rewrite Optimizations in ClickHouse
- Improved Memory Bound Merging and Aggregation Interaction in Query Plans
- Concurrent Merge Scheduling Policy Updated to Round-Robin by Default