v.23.10
Backward Incompatible Changes
8- No Longer an Option to Automatically Remove Broken Data Parts
- Obsolete In-Memory Data Parts Must Be Removed Before Upgrade
- Remove Meilisearch Integration Due to Protocol Change
- Rename directory monitor to background INSERT with backward compatibility
- Do not confuse client-side send_timeout with server-side receive_timeout
- Comparison of Time Interval Units Throws Exception
- Rewritten S3Queue Table Engine with Improved Zookeeper Integration and New System Tables
- Arbitrary paths no longer interpreted as /query endpoint requests
Bug Fix
60- Skip Hardlinking Inverted Index Files in Mutation
- Fixed regex bug in match function with alternation pattern
- Fix 'Cannot find column' in read-in-order optimization with ARRAY JOIN
- Support for Nullable JSON Subcolumns in Queries
- Re-add fix for accurateCastOrNull #54629
- Fix DEFAULT detection for Distributed table columns without AS
- Proper cleanup on exception in ShellCommandSource constructor
- Fix Deadlock in LDAP Assigned Role Update
- Suppress Error Statistics Update for Internal Exceptions #55128
- Fix Deadlock in Backups
- Fix Iceberg Files Retrieval Issue
- Fix partition pruning for extra columns in set
- Fix Skip Index Recalculation in ALTER UPDATE Queries with Adaptive Granularity
- Fix Background Download in FS Cache
- Avoid Memory Leaks in Compressors Due to Missing Buffer Finalization
- Fix execution of functions on sparse columns
- Fix Nested Merge Issue in SummingMergeTree
- Fix detached partition drop bug in replicated merge tree on S3
- Fix crash in MergeSortingPartialResultTransform due to zero chunks after remerge
- Fix data-race in CreatingSetsTransform on errors due to shared exception
- Fix trash optimization issue
- Fix StorageHDFS Leak Issue
- Fix parsing of arrays in cast operator #55417
- Fix Filtering by Virtual Columns with OR Condition in Query
- Fix MongoDB Connection Issues
- Fix MySQL Interface Boolean Representation
- Fix MySQL DateTime Formatting and LowCardinality Types Reporting
- Make use_mysql_types_in_show_columns Affect Only SHOW COLUMNS
- Fix stack symbolizer parsing of DW_FORM_ref_addr to prevent crashes
- Destroy fiber on exception in AsyncTaskExecutor's cancelBefore
- Fix Query Parameters with Custom HTTP Handlers
- Fix Non-handled Data Check for Values Format
- Fix Invalid Cursor State in ODBC with MS SQL Server
- Fix max execution time and break overflow mode
- Fix QueryNormalizer Crash with Cyclic Aliases
- Disable Incorrect Optimization and Add Test
- Merging #52352 and #55621 by Alexey Milovidov
- Add test for correct decimal sorting
- Fix Progress Bar for S3 and Azure Cluster Functions
- Fix filtering by virtual columns with OR filter in query
- Fixes and Improvements for Iceberg Storage
- Fix data race in CreatingSetsTransform (v2)
- Throw Exception for Illegal Float Parsing with Precise Parsing Enabled
- Disable Predicate Pushdown for CTEs with Stateful Functions
- Fix ASTSelectWithUnionQuery to Preserve FORMAT in Queries
- Fix possible segfault in Native ORC input format
- Fix Window Functions for Sparse Columns
- fix: StorageNull supports subcolumns
- Do not log retriable errors for Replicated mutate/merge
- Fix SHOW DATABASES LIMIT N Functionality
- Fix Protobuf Schema with Underscore Fields
- Fix dateTime64ToSnowflake64() for Non-Default Scale
- Fix Arrow Dictionary Column Input/Output
- Fix AvroConfluent Schema Fetching from Schema Registry
- Fix Block Structure Mismatch in Buffer Table During Concurrent ALTER and INSERT
- Fix free space accounting for least_used JBOD policy
- Fix missing scalar issue in subqueries for table functions
- Fix Query Result Error with http_write_exception_in_output_format=1
- Fix schema cache for fallback JSON-JSONEachRow settings
- Add Error Handler to ODBC Bridge
Build/Testing/Packaging Improvement
7- Database Initialization Optimization in Docker to Prevent Infinite Container Restarts
- Resource for Building ClickHouse Without Submodules in Darwin Task
- Error Building ClickHouse with Global AVX Instructions Due to Snappy CRC32 Issue
- Solve issue with launching standalone clickhouse-keeper from clickhouse-server package
- RabbitMQ Updated to 3.12.6 with Enhanced Logs Collection in Tests
- Modified Error Message for OpenSSL and BoringSSL in Functional Test
- Use upstream repo for apache datasketches
Improvement
40- Functions toDayOfWeek, toYearWeek, and toWeek Updated to Support String Arguments
- Introduced date_time_overflow_behavior setting for handling date conversions
- Implement Query Parameters Support for ALTER TABLE ... ACTION PARTITION
- Print Processor IDs in EXPLAIN More Clearly
- Creating Direct Dictionaries with Lifetime Field Rejected
- Allow Parameters in Partition Queries for ALTER TABLE Command
- Add xid Column to system.zookeeper_connection
- Display Correct Server Settings in system.server_settings After Configuration Reload
- Add Support for Mathematical Minus Character in Queries
- Add Replica Groups to Experimental Replicated Database Engine
- Enhance S3 Error Handling by Increasing Default Retry Attempts
- Add hostname_levenshtein_distance load balancing mode
- Improve Hiding Secrets in Logs
- Projection Analysis Updates in Query Plan
- Updated Result Column Naming for Untuple Function with Aliases in ClickHouse
- Added Settings for DESCRIBE Query: Include Virtual Columns and Compact Output Option
- Fix for OPTIMIZE Failure Due to Inconsistent Projections
- Allow Multiple MaterializedPostgreSQL Tables per Postgres Table
- Allow parsing of negative DateTime64 and DateTime with fractional part from short strings
- Improved MySQL Compatibility: New Fields in Information Schema Tables and Columns
- Clickhouse-client fails to display "0 rows in set" when an exception occurs
- Support renaming tables without the TABLE keyword
- Add internal_replication to system.clusters
- Select Remote Proxy Resolver by Request Protocol and Update Documentation
- Avoid Retry on Keeper INSERT after Table Shutdown
- SHOW COLUMNS Enhancements for FixedString and MySQL Type Mapping
- ClickHouse Enhances ReplicatedMergeTree Startup by Restoring Closest Ancestor Parts
- Advanced Dashboard Adds Draggable Charts on Touch Devices
- Use Default Query Format for HTTP Exception Output
- Improve Messaging for Materialized View Pitfalls
- Database Behavior Consistency in ClickHouse with Queries After Drop
- Functions Enhance Compatibility with String-Encoded Date Arguments
- Setting apply_deleted_mask for Debugging Deleted Rows in ClickHouse
- Allow Skipping Null Values When Serializing Tuple to JSON
- Functions addDate and subDate Support String-Encoded Date Arguments for Enhanced MySQL Compatibility
- Allow Unquoted Strings with CR in CSV Format
- Allow clickhouse-keeper to use embedded config
- Set Limit for Max Configuration Value of queued.min.messages in Kafka
- Fix typo in SQL function minSampleSizeContinous to minSampleSizeContinuous for backwards compatibility
- Print Path for Broken Disk Parts Before Server Shutdown
New Feature
23- Add arrayFold function for multi-array lambda accumulation
- Support for Npy Format in ClickHouse
- Enhancement: Indexing with Space-Filling Curves in ClickHouse
- New Setting: force_optimize_projection_name to Validate Projection Usage in Queries
- Support Asynchronous Inserts with External Data via Native Protocol
- Added lttb Aggregation Function for Data Downsampling
- Improved CHECK TABLE Performance and Usability in ClickHouse
- Added jsonMergePatch Function for Merging JSON Strings
- Second Part of Kusto Query Language Dialect Support Merged
- Added arrayRandomSample SQL Function for Convenient Element Sampling
- Introduce -ArgMin/-ArgMax Aggregate Combinators and Reorganize Combinators Folder
- Allow dropping cache for Protobuf format with SYSTEM DROP SCHEMA FORMAT CACHE
- Add External HTTP Basic Authenticator
- Added byteSwap Function for Reversing Unsigned Integer Bytes
- Added SQL Query Formatting Functions
- Added DWARF Input Format for ELF Debug Symbols
- Allow Saving of Unparsed Records and Errors in RabbitMQ, NATS, and FileLog Engines
- Keeper client enhancement: add get_all_children_number command to count children nodes under a specific path
- Keeper Client Enhancement: Add get_direct_children_number Command
- Add simpler `SHOW SETTING` statement to ClickHouse
- Added substreams and filenames fields to system.parts_columns table
- Add Support for SHOW MERGES Query
- Introduce create_table_empty_primary_key_by_default setting for default ORDER BY ()
Performance Improvement
26- Add option to preserve stream count after window functions for parallel processing
- Release Additional Streams for Small Data
- RoaringBitmaps Optimization Before Serialization
- Optimized Space Consumption for Inverted Indexes through Bitmap Representation
- Fix Context Lock Contention for Better Performance in Concurrent Queries
- Improved Inverted Index Creation Performance by 30% with absl::flat_hash_map
- Support ORC Filter Push Down at Rowgroup Level
- Improve Performance of External Aggregation with Temporary Files
- Set Default Marks Cache Size for Secondary Indices
- Avoid Unnecessary Reconstruction of Index Granules in Skip Indexes
- Cache CAST function to enhance IN performance with mismatched set element types
- Performance Enhancement for ColumnVector Insertion Methods
- Optimized Map Subscript Operations for Improved Key Position Prediction
- Support pruning of struct fields in Parquet
- Add Tuning for Parallel Replicas in Query Execution
- Optimized Memory Consumption for Temporary File Aggregation
- Distributed Queries in Async Socket Mode Respect Max Threads Limit
- Caching Skip-able Entries in Zookeeper Distributed DDL Queue
- Experimental Inverted Indexes Introduce Simplified Row ID Storage Heuristic
- Improve Write Performance for EmbeddedRocksDB Tables
- Improved ClickHouse Resilience for Large Number of Partition Parts
- Reduced Memory Consumption for Hierarchical Dictionaries
- All Dictionaries Support Asynchronous Executor Setting
- Prevent Excessive Memory Usage in AggregateFunctionTopKGenericData
- High CPU Usage in Keeper Due to AsyncMetrics Threads Processing Heavy Watches
- Add optimize_trivial_approximate_count_query setting for EmbeddedRocksDB and StorageJoin