v.23.9
Backward Incompatible Changes
3Bug Fix
55- Fix Zero-Copy Replication for REPLACE/MOVE PARTITION
- Fix Zero-Copy Locks with Hardlinks
- Fix Zero-Copy Garbage in Experimental Replication Feature
- Pass HTTP Retry Timeout as Milliseconds
- Fix misleading OUTFILE error message in CapnProto/Protobuf
- Fix summary reporting with parallel replicas using LIMIT
- Fix BACKUP throttling for S3 and other scenarios
- Fix IO Throttling During Directory Copying
- Fix Column Loss in Prewhere Condition Actions
- Fixed internal error in byte-equal parts replacement
- Fix Required Columns in Interpolate Expression
- Fix cluster discovery initialization and configure fail points
- Fix issues in accurateCastOrNull #54136
- Fix nullable primary key with FINAL modifier
- Fixed Insertion Error in Replicated Materialized View with Duplicated Data
- Fix Bloom Filter to Support IPv6
- fix type mismatch with IPv4
- Fix system.data_skipping_indices for recreated indices
- fix name clash for multiple join rewriter v2
- Fix errors in system.errors post PR #54306
- Fix isZeroOrNull function to handle NULL values
- Fix parallel replicas with prefer_localhost_replica = 1
- Fix logical error in vertical merge and optimize cleanup
- Fix URI Validation Error in S3 Table Function
- Fix segfault in AST optimization of arrayExists function
- Check for Overflow Before Addition in analysisOfVariance Function
- Reproduce and Fix Bug in removeSharedRecursive #54430
- Fix Incorrect Result with SimpleAggregateFunction in PREWHERE and FINAL
- Fix filtering parts with indexHint for non-analyzer
- Fix aggregate projections with normalized states
- clickhouse-local support for multiquery parameter #54498
- clickhouse-local adds support for --database command line argument
- Fix Parsing Error in -WithNames Formats with Disabled Header Input Format
- Fix CHECKSUM_DOESNT_MATCH Error
- Fix Sorting of UNION ALL with Sorted Results
- Fix snapshot install in Keeper
- Fix race condition in ColumnUnique
- Annoy/Usearch index: Fix LOGICAL_ERROR during build-up with default values
- Fix serialization of ColumnDecimal
- Fix Schema Inference for Cluster Functions with Column Names Containing Spaces
- Fix insertion table structure handling for defaults and explicit columns
- Fix regex match condition in ClickHouse code
- Fix ReplacingMergeTree with Vertical Merge and Cleanup
- Fix incorrect values in virtual columns after ORDER BY
- Fix indexHint filtering for non-analyzer parts
- Fix Keeper Segfault During Shutdown
- Fix Invalid Number of Rows in Chunk in MaterializedPostgreSQL
- Move Obsolete Format Settings to Separate Section
- Rebuild minmax_count_projection on Partition Key Modification
- Fix Bad Cast to ColumnVector<Int128> in if Function
- Prevent Attaching Parts from Different Projections or Indices
- Store NULL in Scalar Result Map for Empty Subquery Result
- Fix FINAL for Invalid Read Ranges Issue
- Fix quorum insertion without keeper retries
- Fix nullable state in ClickHouse
Build/Testing/Packaging Improvement
7- Bump ClickHouse Compiler from Clang 16 to 17
- Regenerated TLD Data for Lookups in tldLookup.generated.cpp
- Remove redundant clickhouse-keeper-client symlink
- Use /usr/bin/env for Bash support on Nix OS
- CMake adds PROFILE_CPU option for perf record without DWARF call graph
- If Linker Is Not LLD, Trigger Fatal Error
- Replaced Turbo-Base64 with aklomp-base64 for base64 encoding/decoding in ClickHouse
Experimental Feature
2Improvement
44- Allow File Name Hashing in MergeTree Data Parts to Prevent Length Errors
- Parse JSON as JSONEachRow when metadata parsing fails
- Output Exception as JSON/XML in HTTP Queries with Configurable Setting
- New Field 'data_length' Added to information_schema.tables for Disk Size Approximation
- MySQL Interface Gains Minimal Prepared Statements for Tableau Online Connection to ClickHouse
- Support case-insensitive and dot-all matching in regexp_tree dictionaries
- Keeper Improvement: Add createIfNotExists Command
- More Precise Integer Type Inference Fix
- Introduced Charset Resolution in String Literals for MaterializedMySQL
- Fix issue with EmbeddedRocksDB table engine not closing files on NFS after DROP TABLE
- RESTORE TABLE ON CLUSTER Requires Matching UUIDs for Replicated Tables
- Added restore setting restore_broken_parts_as_detached to handle broken parts during RESTORE process
- Add elapsed_ns field to HTTP headers X-ClickHouse-Progress and X-ClickHouse-Summary
- Implementation of reconfig, sync, and exists commands for keeper-client
- clickhouse-local and clickhouse-client Enhance --query Parameter Support
- Add Sub-Second Precision to formatReadableTimeDelta
- Enable allow_remove_stale_moving_parts by default
- Fix Cache Count Usage and Enhance Progress Bar for Archive Reading
- Add S3 Credential Support via SSO with AWS_PROFILE Variable
- Support NULL as Default for Nested Types in Input Formats
- Allow reading unusual chunk configurations from Arrow/Parquet formats
- Add STD alias to stddevPop function for MySQL compatibility
- Add addDate and subDate functions for MySQL compatibility and consistency
- Add modification_time to system.detached_parts
- Added setting to control inclusion of remaining string in splitBy functions
- Better Integer Types Inference for Int64/UInt64 Fields, Including Nested Types and Functions
- Added scalar array operations for multiplication, division, and modulo
- Add optional version argument to rm command for safer deletes in keeper-client
- Disable Server Shutdown by Systemd to Prevent Data Loss
- Added is_deterministic Field to system.functions Table
- Enhancements to ClickHouse's Information Schema for MySQL Compatibility
- Query Cache Error for Non-Deterministic Functions Enhances User Clarity
- Forbid Special Columns for File and URL Storages and Fix Inserts into Ephemeral Columns
- More Configurable Metadata Collection for Backup
- ClickHouse Local Log Files Now Include Timestamps and Thread IDs
- Field is_obsolete in system.merge_tree_settings table now indicates obsolete settings
- Enhance INTERVAL Literals to Support Plural Forms
- Always Allow Creation of Projection with Nullable PK
- Retry S3 Operations After Connection Reset Failure
- Make Exception Message Clear for Invalid Settings Range
- LIKE and Regular Expressions Now Support Non-UTF-8 Patterns with Binary Matching
- Added ContextLockWaitMicroseconds Profile Event
- The Keeper Dynamically Adjusts Log Levels
- Added timestamp function for MySQL compatibility
New Feature
20- Improve JSON Schema Inference and Type Handling in ClickHouse
- Added IO Scheduling Support for Remote Disks in ClickHouse
- Added SSH Key Authentication for Native TCP Protocol
- Added _block_number column for MergeTree tables
- Add IF EMPTY clause for DROP TABLE queries
- SQL Functions Support Non-Constant Timezone Arguments
- Add Support for ALTER TABLE MODIFY COMMENT in ClickHouse
- Added GCD Codec for Data Compression in ClickHouse
- New Type Aliases for DECIMAL in ClickHouse Enhance MySQL Compatibility
- Added backup_log table for tracking BACKUP and RESTORE operations
- Added Output Format Setting for Escaping Special Characters in Markdown
- Add decodeHTMLComponent function
- Added peak_threads_usage to query_log table
- Add SHOW FUNCTIONS support to clickhouse-client
- Added toDaysSinceYearZero Function with MySQL Compatibility and Enhanced DateTime Support
- Added Date Conversion Functions for Integer Encoded Dates
- Add String Distance Functions: byteHammingDistance, editDistance
- Allow Custom Expiration Date for User Credentials
- Allow S3-style URLs for table functions with automatic HTTP conversion
- Add print_pretty_type_names setting for better nested type representation
Performance Improvement
19- Speed up S3 reading with default prefetches
- Do not implicitly read PK and version columns in lonely parts for FINAL queries
- Optimize Group By for Constant Keys in ClickHouse
- Improve Sorting and Insertion Performance for Decimal Columns in MergeTree
- Improve performance for large query analysis, fixes #51224 and #51469
- Optimization for COUNT(DISTINCT) and uniq variants in subqueries with GROUP BY
- Remove manual memory management and delegate to jemalloc for performance boost
- Fixed high CPU consumption issue with NATS
- Performance Improvements in toString Execution for Non-Datetime Arguments
- Improving JSON Serialization by Directly Using ColumnString
- Enable ORDER BY Optimization for MergeTree Tables Behind a View
- Improve JSON SQL Functions by Reusing GeneratorJSONPath
- Keeper Optimizes Request Performance with Batch Flushing
- clickhouse-client now processes files in parallel with INFILE 'glob_expression'
- Allow Primary Key Usage in IN Function with Different Column Types
- Hash JOIN Optimizes Internal Buffer Usage in ClickHouse
- Respect max_block_size for Array Join to Prevent OOM Issues
- Reuse HTTP Connections in s3 Table Function
- Replace Linear Search with Binary Search in MergeTreeRangeReader