v.23.7
Backward Incompatible Changes
5- Add NAMED COLLECTION Access Type (Backward Incompatible)
- Fix typo in system.parts column name last_removal_attempt_time
- Bump distributed_ddl_entry_format_version to 5 by default for OpenTelemetry support
- Check Projection Metadata to Prevent Server Startup Issues
- Experimental feature hashid removed due to bugs and implementation quality concerns
Bug Fix
47- Fix syncTables for MaterializedPostgreSQL
- Fix Projection with Optimize Aggregators of Group By Keys
- Fix optimize_skip_unused_shards with JOINs
- Fix formatDateTime() for fractional negative datetime64
- Functions hasToken* Incorrect; Test Addition for Issue #43358
- Fix function placement optimization before sorting
- Fix Block Structure Mismatch in Pipe::unitePipes for FINAL
- Fix SIGSEGV for Zero-Weight Clusters in INSERT INTO FUNCTION
- Fix Timeout for Hedged Requests
- Fix ANTI Join Logical Error with NULL
- Fix Moving 'IN' Conditions to PREWHERE
- Do not apply PredicateExpressionsOptimizer for ASOF/ANTI join
- Fix Async Insert with Deduplication for ReplicatedMergeTree
- Fix empty column handling in parseSipHashKey
- Fix segfault for invalid EmbeddedRocksdb table creation
- Fix MongoDB Inserts
- Fix Deadlock on DatabaseCatalog Shutdown
- Fix Subquery Operators Error
- Fix async connection to hosts with multiple IPs
- Do not remove inputs after ActionsDAG::merge
- Check refcount in RemoveManyObjectStorageOperation::finalize instead of execute
- Allow Parametric UDFs Implementation
- Small fix for toDateTime64() for dates after 2283-12-31
- Fix ORDER BY in WINDOW functions
- Fix Incorrect Projection Analysis with Monotonic Function Aggregations
- Fix error in groupArrayMoving function
- Disable Direct Join for Range Dictionary
- Fix sticky mutations test and rare race condition
- Fix race condition in Web disk #52211
- Fix data race in Connection::setAsyncCallback for unknown server packet
- Fix temp data deletion on startup and add test
- Don't Use Minmax Count Projections for Nullable Column Counting
- MergeTree/ReplicatedMergeTree Should Use Server Timezone for Log Entries
- Fix Parameterized View with CTE and Multiple Usage
- Disable Expression Templates for Time Intervals
- Fix apply_snapshot in Keeper
- Update build-osx.md by AlexBykovski
- Fix countSubstrings Hang with Empty Needle in Column Haystack
- Fix Normal Projection with Merge Table
- Fix double-free issue in Aggregator
- Fixed Buffer Engine Insertion Issue
- AnyHash Implementation Non-Conformance
- Check Recursion Depth in OptimizedRegularExpression
- Fix data-race in DatabaseReplicated startupTables and canExecuteReplicatedMetadataAlter
- Fix Abort in Transform Function
- Fix lightweight delete after projection drop
- Fix "Cannot drain connections: cancel first" Error
Build/Testing/Packaging Improvement
8- Add Experimental ClickHouse Builds for Linux RISC-V 64 to CI
- Add Integration Test for Enabled Analyzer
- Reproducible Builds for Rust
- Update Cargo Dependencies in ClickHouse
- Enhance CHColumnToArrowColumn to Support Nullable Arrays for Gluten
- CCTZ Library Updated to Master with No User-Visible Changes
- system.licenses table now includes Poco library
- Check for Bad Punctuation in Text
Experimental Feature
11- Writing Parquet Files Now 10x Faster with Multi-threading
- Added Support for PRQL Query Language
- Allow Custom Disk Naming for Disks in ClickHouse
- Fixed crash in MaterializedMySQL when using disconnected mysqlxx::Pool::Entry
- Experimental Support for CREATE TABLE AS SELECT in MaterializedMySQL
- Experimental MaterializedMySQL: Automatic Text Type Conversion to UTF8
- Experimental MaterializedMySQL Supports Unquoted UTF-8 Strings in DDL
- Experimental MaterializedMySQL Now Supports Double Quoted Comments
- Upgrade Intel QPL and accel-config; Fix IOTLB miss performance issue for IAA accelerators
- session_timezone setting demoted to experimental in version 23.6
- Support Incremental Reconfiguration with ZooKeeper Command in ClickHouse Keeper
Improvement
42- Use read_bytes/total_bytes_to_read for Progress Bar in S3/File/URL Table Functions
- Introduce wait_for_unique_parts_send_before_shutdown_ms and Fix Shutdown Order for Tables and Handlers
- Allow SQL Standard FETCH without OFFSET
- Allow Filtering of HTTP Headers in URL/S3 Table Functions with http_forbid_headers Configuration
- Don't Log Messages About 16 EiB Free Space
- Properly check limit for sleepEachRow function and add function_sleep_max_microseconds_per_block setting
- Fix Issues in geoHash Functions
- Log Async Insert Flush Queries into system.query_log
- Functions date_diff and age Enhanced for Microsecond Precision
- Improve Path Parsing in ClickHouse Keeper Client
- Fix heap overflow bug in third-party product using ClickHouse while reading from HDFS
- Add Option to Disable Native Copy for S3 in Backup/Restore Settings
- Add primary_key_size column to system.parts table
- Allow clickhouse-local to run without procfs, home directory, and glibc name resolution plugins
- Add %a Placeholder for Full Filename in rename_files_after_processing Setting
- Add modification_time column to system.parts_columns
- Add input_format_csv_use_default_on_bad_values setting for default value on CSV parsing errors
- Added Crash Log Flush to Disk After Unexpected Crash
- Fix Dashboard Error Display and Chart Overlap Issues
- Allow UUID to UInt128 Conversion
- Added Nullable Arguments Support for range Function
- Convert toyear Condition to Range Format
- Improve MySQL Compatibility for SHOW INDEX Statement
- Fix use_structure_from_insertion_table_in_table_functions for MATERIALIZED and ALIAS columns
- Cache Dictionary Requests Unique Keys from Source
- Fixed EXPLAIN Query Settings Not Applying with FORMAT
- Allow SETTINGS Before FORMAT in DESCRIBE TABLE for Compatibility with SELECT Query
- Var-Int Encoding Updated to Support Full 64-Bit Range
- Update Certificates Automatically Without Manual System Reload
- Added allow_create_index_without_type Setting for INDEX Queries
- Log Messages Logged to system.text_log on Server Startup
- Session creation robust to unreachable IP addresses in HTTP endpoints
- Avro Input Format Supports Union with Single Type
- Add optimize_use_implicit_projections Setting to Disable Implicit Projections
- Function hasToken no longer allows infinite loops, resolving issue #52156
- Create ZK Ancestors Optimistically (#52195)
- Fix Not Found Column Error in ClickHouse Reading In-Order and Constants
- Check Early for Invalid S2 Geo Primitives in ClickHouse
- Add Missing Projection QueryAccessInfo for Optimized Query Plans
- Enhance Error Reporting in ZooKeeperRetriesControl by Preserving Original Stack Trace
- Wait for Zero Copy Replication Lock on All Disks
- Interserver Port Closure After Table Shutdown
New Feature
30- Added New Database Engines: Overlay, Filesystem, S3, and HDFS
- Add External Disk Support in Keeper for Snapshots and Logs
- Add multi-directory selection support for globs
- Kafka connector supports fetching Avro schema with basic authentication from schema registry
- Add arrayJaccardIndex function for Jaccard similarity computation
- Add is_obsolete Column to system.settings Table
- Implement Support for Encrypted Elements in Configuration File
- Grace Hash Join Algorithm Enhancements for FULL and RIGHT JOINs
- Add SYSTEM STOP LISTEN query for graceful termination
- Add input_format_csv_allow_variable_number_of_columns option
- Another boring feature: add substring_index function like Spark or MySQL
- System Table jemalloc_bins for Memory Allocation Stats
- Add RowBinaryWithDefaults format with default value flags
- Added default_temporary_table_engine setting for temporary tables
- Added initcap and initcapUTF8 Functions for Capitalization
- Create Table Enhanced with PRIMARY KEY Syntax in Column Definition
- Added Date and Time Format Specifiers for Log File Names
- Added Peak Memory Usage Statistic to HTTP Headers
- Added hasSubsequence functions for string matching
- Add array_agg as alias for groupArray to enhance PostgreSQL compatibility
- Add any_value as compatibility alias for any aggregate function
- Add array_concat_agg function for BigQuery compatibility
- Add OCTET_LENGTH Alias for length
- Added firstLine function for extracting first line from multi-line string
- Implement KQL-style Formatting for Interval Data Type Compatibility
- Added SYSTEM FLUSH ASYNC INSERT QUEUE Query and Server-Side Settings
- Aliases for current_database and new function current_schemas for PostgreSQL compatibility
- Add Function Aliases for today and now in ClickHouse
- Support async_deduplication_token for async insert
- Add disable_url_encoding setting to URL engine
Performance Improvement
15- Enable Default Sparse Serialization Format for Improved Performance
- Enable default settings for move_all_conditions_to_prewhere and enable_multiple_prewhere_read_steps
- Improves Query Performance by Tuning Allocator
- Fixed-size tasks in MergeTreePrefetchedReadPool and S3 connection pooling improvements
- More Pushdown Optimization in Join Operations
- Improve grace_hash join by reserving hash table size
- Improving OpenedFileCache Lock Contention with Sharded Sub-Maps
- Move PK Column Conditions to End of PREWHERE Chain in ClickHouse
- Speed Up COUNT(DISTINCT) for String Types by Inlining SipHash with 11.6% QPS Improvement in Query Q8
- Enable allow_vertical_merges_from_compact_to_wide_parts by default to save memory during merges
- Fix Projection Analysis Affecting Primary Key Validity
- Reduce Syscalls in FileCache::loadMetadata for Faster Server Startup
- Allow Strict Lower Boundary for File Segment Size with Background Data Downloading
- Decreased Default Timeouts for S3 and HTTP Requests
- New Setting: merge_tree_determine_task_size_by_prewhere_columns Added