v.23.8
Backward Incompatible Changes
5- Dynamic Disk Naming Changes in Disk Function Arguments
- ClickHouse Benchmark Update: Concurrent Connections and QPS Calculation Improvements, JSON Output Option Removed
- Removal of Redundant Time Columns from system.text_log and system.metric_log
- Deprecate and Remove Metadata Cache Feature
- Disable 3DES Support in TLS Connections
Bug Fix
63- Do not reset Annoy index during multi-mark build-up
- Fix Temporary Directory Usage During RESTORE
- Fix Nullable(IPv4) Binary Arithmetic
- Support for IPv4 and IPv6 Data Types as Dictionary Attributes
- Fix for Checksum of Compress Marks
- Fix Comma Parsing in CSV Datetime Handling
- Don't throw exception for executable UDF with parameters
- Fix Skip Indexes and Projections in ALTER DELETE Queries
- MaterializedMySQL: Resolve Infinite Loop in ReadBuffer::read
- Load Suggestion with ClickHouse Dialect
- Init and Destroy Ares Channel on Demand
- Fix Filtering by Virtual Columns with OR Expression
- Fix crash in tuple function with one sparse column argument
- Fix named collections on cluster #52687
- Fix unnecessary column reading in multistage PREWHERE
- Fix multi-column sort behavior with nulls first direction
- Fix Data Race in Keeper Reconfiguration
- Fix sorting of sparse columns with large limit
- clickhouse-keeper: fix server implementation with poll
- Make RegExp Analyzer Support Named Capturing Groups
- Fix assert issue in PushingAsyncPipelineExecutor in clickhouse-local
- Fix Reading of Empty Nested LowCardinality Arrays
- Added Tests for Session Log and Resolved Login/Logout Inconsistency
- Fix password leak in show create MySQL table
- Convert Sparse Column Format to Full in CreateSetAndFilterOnTheFlyStep
- Fix race condition in fs cache during empty key prefix directory deletion
- Fix ZstdDeflatingWriteBuffer Output Truncation Issue
- Fix query_id in part_log for async flush queries
- Fix Cache Error: "Read Unexpected Size"
- Disable New Parquet Encoder #53130
- Fix Not-ready Set Exception in ClickHouse
- Fix character escaping in PostgreSQL engine
- Experimental session_log Updates: Tests Added and Inconsistencies Fixed
- Fix for Sub-Second Intervals in DateTime
- Fix Context Expired Error in Dictionaries
- Fix AST Format for Normal Projection
- Forbid use_structure_from_insertion_table_in_table_functions in Scalar execution
- Fix lazy database loading in system.table select query
- Fixed data skipping indices for MaterializedMySQL
- Fix Handling of Single Carriage Return in TSV Segmentation Engine
- Fix handling of Context expiration error
- Fix timeout_overflow_mode for subquery in IN clause
- Fix Unexpected Behavior in Issue #53152
- Fix JSON_QUERY Function Parse Error for Numeric Paths
- Fix Column Order for Parallel FINAL Queries
- Fixed SELECTing from ReplacingMergeTree with do_not_merge_across_partitions_select_final
- Flush Async Insert Queue on Shutdown
- Fix crash in join on sparse column
- Fix UB in Set Skipping Index for Functions with Incorrect Args
- Fix UB in Inverted Indexes (Experimental Feature)
- Fix: Interpolate Expression Uses Source Column Instead of Aliased Name
- Fix Dropped Granules in EXPLAIN PLAN for Index 1
- Correct Handling of Totals and Extremes with DelayedSource
- Prepared set cache issue in mutation pipeline
- Fix JSON Subcolumns Bug in UPDATE and DELETE Queries
- Fix Filter Pushdown for Full Sorting Merge Join
- Fix NULL LowCardinality NOT IN Bug
- Fix sorted distinct with sparse columns
- transform: fix default column handling for multiple rows
- Fix fuzzer crash in parseDateTime
- MaterializedPostgreSQL: Fix uncaught exception in getCreateTableQueryImpl
- Fix PostgreSQL engine segfault issue
- Fix named_collection_admin alias #54066 by Kseniia Sumarokova
Build/Testing/Packaging Improvement
20- Don't Expose ClickHouse Binary Symbols to Dynamic Linker
- Add clickhouse-keeper-client symlink to clickhouse-server package
- Add sqltest to CI for SQL 2016 conformance reporting
- Upgrade PRQL to 0.9.3
- System Tables Exported to ClickHouse Cloud
- Compiler Profile Data Uploaded to ClickHouse Cloud
- Speed Up Debug and Tidy Builds
- Speed Up Build by Removing Unused Headers
- Remove Excess Garbage from Codebase
- Function arrayAUC Refactored to Remove Heavy C++ Templates
- Some translation units always rebuilt; issue fixed.
- Compiler's Profile Data Uploaded to ClickHouse Cloud After Previous Attempt
- Export CI Logs to ClickHouse Cloud for Stateful Tests
- Export CI Logs in Stress Tests
- Export CI Logs in Fuzzer
- Preserve environment parameters in clickhouse start command
- Follow-up on #53418: Improvements to install_check.py and ENV parameter tests on init.d start
- Reorganize CMake File Management to Avoid Duplications
- Upgrade snappy to 1.1.10
- Slight CMake Build Improvements: Dependency Sanitization and Duplicate Removal
Experimental Feature
3Improvement
70- Add stderr_reaction Configuration to Manage External Command stderr Handling
- Add partition column to system part_log and merge table
- Dynamic Runtime Configuration of Cache Sizes in ClickHouse
- Automatically Select "Complex Key" Layout for Dictionaries with Complex Keys
- Add use_concurrency_control setting for improved concurrency testing
- Added suggestions for mistyped database and table names
- Performance Comparison of Reading Small Files from HDFS with Gluten vs. Spark
- Excessive Error Logs After Session Expiration
- Introduce Time-Bound Fallback ZooKeeper Sessions and Fix DNS Index Column
- Add logging for max_partitions_per_insert_block limit reached
- Added Custom Commands to ClickHouse Keeper Client for Easier Debugging
- Updated Azure Blob Storage Connection String Handling
- Fix description for filtering sets in full_sorting_merge JOIN algorithm
- Fixed High Memory Consumption in Aggregator with Large max_block_size
- Add SYSTEM SYNC FILESYSTEM CACHE Command for State Comparison and Fixing in ClickHouse
- Attempt to Create Generic Proxy Resolver for CH with Backwards Compatibility
- Support for Reading Tuple Subcolumns from Various Storage Systems
- Function arrayIntersect Returns Values in Order of First Argument
- Add Queries for Managing Access Entities in Access Storage
- Make ALTER TABLE FREEZE Queries Non-Replicated in Replicated Database Engine
- Added system table flushing on unexpected shutdown
- Fix s3 table function compatibility with pre-signed URLs
- Add Alias Column 'name' to system.events and system.metrics Tables
- Added No-Op Support for CREATE UNIQUE INDEX in SQL Parser
- Add Predefined Macros Support in Kafka Engine Settings
- Disable Filesystem Cache Update During Backup/Restore
- S3 Endpoint Configuration Enhancements
- Enhance clickhouse-local with Positional Options and Global UDF Settings
- system.asynchronous_metrics adds QueryCacheEntries and QueryCacheBytes metrics
- Added s3_storage_class Parameter to BACKUP Statement for S3 Backups
- Add utility print-backup-info.py for parsing and printing backup metadata information
- Simplifying Case Sensitivity in ClickHouse's Information Schema
- Improved Performance and Usability for CHECK TABLE Query
- Add modulo and division support for tuples in ClickHouse
- Search for Default YAML and YML Configs in ClickHouse Client After XML
- Merging Configs with Different Root Node Names Bypasses without Exception
- New Memory Profiler Features: Min/Max Allocation Size Tracking
- Add precise_float_parsing setting for float parsing methods
- Use Consistent Default Paths for ClickHouse Keeper Symlink and Executable
- Improve error message for remote table function
- Added Custom Storage Policy Support in RESTORE Queries
- Add S3 Request Throttling for Backup Operations
- Add settings to ignore ON CLUSTER clause in replicated function management
- EXPLAIN Actions for JOIN Step
- Make hasTokenOrNull and hasTokenCaseInsensitiveOrNull Return Null for Empty Needles
- Allow Restriction of Allowed Paths for Filesystem Caches
- Added Custom Commands for Easier ClickHouse Debugging
- Add Diagnostic Info for File Name in Schema Inference
- Client Fallback to Main Connection for Suggestions Without Session Creation
- Add EXCEPT Clause to SYSTEM STOP/START LISTEN QUERIES Command
- Change Default max_concurrent_queries from 100 to 1000
- Limit Concurrent Background Partition Optimize Merges
- Added Setting to Ignore Directory Exists Error During Database Replication
- Server Settings Misconfiguration Now Fails Gracefully
- ClickHouse Server Respects cgroup Memory Limits on Configuration Reload
- Add Option to Disable Flush for Distributed Tables on DETACH, DROP, or Shutdown
- domainRFC Function Adds IPv6 Support
- Use Longer Timeout for S3 CopyObject in Backups
- Added Server Setting to Limit Array Size for groupArray Function
- SCHEMA Alias Added for DATABASE to Enhance MySQL Compatibility
- Add Asynchronous Metrics for MergeTree Tables in System Database
- SQL Editor in Play UI and Dashboard Disables Grammarly
- Expert Configuration for Cache Size and Policy in ClickHouse
- Added client info validation to TCPHandler query packet
- Retry Loading Parts on Azure in Case of Network Errors
- Stacktrace Propagation for Materialized View Exceptions
- Keeper Client Connection String Search in ClickHouse Config
- Add PartsLockMicroseconds Profile Event to MergeTree Table Engine
- Make RAFT reconnect limit configurable for quicker peer connections
- Ignore Foreign Keys in Table Definitions for MySQL Compatibility
New Feature
24- Direct Import from Archive Formats in ClickHouse
- Add ptr column to system.trace_log for MemorySample and implement flameGraph function
- Added azureBlobStorageCluster table function similar to s3Cluster
- Allow cluster commands without table name in issue #50808
- A system table for monitoring Kafka consumers
- Added max_sessions_for_user Setting
- New toUTCTimestamp and fromUTCTimestamp Functions Added for ClickHouse
- Add CapnProto/Protobuf Schema Conversion Functions to ClickHouse
- New query_cache_usage field in system.query_log shows query cache usage
- Add startsWithUTF8 and endsWithUTF8 functions
- Allow Variable Number of Columns in TSV/CustomSeparated/JSON Input Formats
- Added SYSTEM STOP/START PULLING REPLICATION LOG Queries for ReplicatedMergeTree Testing
- Allow Execution of Constant Non-Deterministic Functions in Mutations
- ERROR while parsing JSON
- Add tupleConcat function, closes #52759, PR #53239
- Support TRUNCATE DATABASE operation
- Add max_threads_for_indexes Setting to Limit Primary Key Processing Threads
- Re-add SipHash Keyed Functions
- Added array manipulation functions: arrayRotateLeft, arrayRotateRight, arrayShiftLeft, and arrayShiftRight.
- Add `name` column as alias to `system.clusters`
- Advanced Dashboard Supports Mass Editing Features
- Advanced Dashboard Enhancements: Maximize and Move Charts
- Added Support for Array Addition and Subtraction in ClickHouse
- Add Support for String Literals as Table Names
Performance Improvement
19- Parquet Filter Pushdown Optimization
- Optimize Batching of Small Row Groups in Parquet
- Optimize Count Handling for Various Input File Formats
- Use File/Path Filter Before Reading in Table Functions
- Enable JIT Compilation for AArch64, PowerPC, SystemZ, RISC-V
- Add rewrite_count_distinct_if_with_count_distinct_implementation setting for countDistinctIf rewrite
- Speed up merging of uniq and uniqExact functions by parallelizing state conversion
- Optimize Performance of Nullable String Keys in Aggregation
- Add Time Filter Optimization Pass in Analyzer for Improved Performance
- Optimize merge for single-level hash sets in uniqExact function
- Join table engine: avoid cloning hash join data structure with all columns
- Implement Native ORC Input Format for Improved Performance
- Dashboard Feature: Data Compression for Large Time Frames on Slow Connections
- Better Thread Pool Utilization for BACKUPs and RESTOREs
- Load Filesystem Cache Metadata in Parallel on Startup
- Improve move_primary_key_columns_to_end_of_prewhere
- Improved Memory Management for ClickHouse Keeper Watch Callbacks
- Cache Row Count in File Functions with Optional Toggle
- Improved Thread Management Boosts S3 Table Function Speed by Over 25%