v.23.3
Bug Fix
61- Fix formats parser resetting and test bad message processing in Kafka
- Fix data size calculation in Keeper
- Fixed Bug in Automatic Retries for DROP TABLE with ReplicatedMergeTree and Atomic Databases
- Fix alias recursion in query normalization
- Fix IPv4/IPv6 Serialization in Binary Formats
- ActionsDAG: Preserve 'and' Result During Optimization
- Improve Query Cancellation on Client Termination
- Fix Arithmetic Operations in Aggregate Optimization
- Fix clickhouse-local abort on JSONEachRow schema inference
- Fix for Expired Role Change
- Fix PREWHERE Column Accumulation in Multiple Steps
- Use Initial Range for Fetching File Size in HTTP Read Buffer
- Fix Progress Bar Issue in URL Tables
- Fix MSan Report in maxIntersections Function
- Fix Bug in Map Data Type
- Fix LIKE search results for quoted non-quotable characters
- Fix abort issue with WITH FILL for empty blocks in Filling Transform
- Fix Date and Integer Inference from String in JSON
- Fix bug in zero-copy replication disk choice during fetch
- Fix typo in systemd service definition
- Fix NOT_IMPLEMENTED Error with CROSS JOIN and Algorithm = Auto
- Fix 'ReplicatedMergeTree' Insertion Issue with 'InMemory' Part Type
- External dictionaries: Fix "unknown library method 'extDict_libClone'" error
- Fix race condition in grace hash join with limit
- Fix PREWHERE Support for Concrete Columns
- Fix possible deadlock in Query Status
- Forbid Insert Select on Same Join Table to Prevent Deadlock
- Skip Merged Partitions for min_age_to_force_merge_seconds Merges
- Modify find_first_symbols for expected behavior with find_first_not_symbols
- Fix big numbers inference in CSV
- Disable Logical Expression Optimizer for Aliased Expressions
- Fix error in decodeURLComponent
- Fix Explain Graph with Projection
- Fix Query Parameters in ClickHouse #47488
- Parameterized view bug fix #47495
- Fuzzer for Data Formats and Fixes by Alexey Milovidov
- Fix DateTime64 Monotonicity Check
- Fix block structure mismatch for Nullable LowCardinality column
- Proper bug fix for Apache Parquet #45878
- Fix BSONEachRow Parallel Parsing for Invalid Document Sizes
- Preserve error in system.distribution_queue on SYSTEM FLUSH DISTRIBUTED
- Check for Duplicate Column in BSONEachRow Format
- Fix zero copy lock wait during move
- Fix aggregation by partitions in ClickHouse
- Fix Bug in Tuple Serialization for BSONEachRow Format
- Fix crash in polygonsSymDifferenceCartesian #47702
- Fix File Reading of Compressed Files with zlib and gzip
- Improve empty query detection in PostgreSQL for pgx golang driver
- Fix DateTime Monotonicity Check for LowCardinality Types
- Use restore_threads for RESTORE ASYNC
- Fix DROP COLUMN in ReplicatedMergeTree with Projections
- Fix Replicated Database Recovery Issue
- Hotfix for Excessive HTTP Warnings #47903
- Fix Field Value Too Long in catboostEvaluate
- Fix Watchdog Exit Code Issue #36971
- Fix for unexpectedly long index file issue
- Fix MaterializedPostgreSQL Query for Replica-Identity Attributes
- Fix signed integer overflow in parseDateTime()
- Use Unique Names for Avro Records to Avoid Schema Reuse
- Correct TCP/HTTP Socket Timeout Configuration in Keeper
- Fix null pointer exception in Avro format
Build/Testing/Packaging Improvement
13- ClickHouse Now Supports C++23
- Fuzz EXPLAIN Queries in the AST Fuzzer
- Split Stress Test and Automated Upgrade Check Implementation
- Updated Docker Ubuntu Image to Address Security Reports
- Adds prompt for overwriting existing ClickHouse binary during curl installation
- Fix server startup error on old distros and ARM related to glibc 2.28 symbols
- Prepare for clang 16 Update
- Added CI Check for ClickHouse Compatibility with Older glibc on ARM
- Add Style Check for Correct NDEBUG Macro Usage
- Speed up the build process
- Bump vectorscan to 5.4.9
- Add Unit Test for Apache Arrow's Fatal Logging Behavior
- Restore native macOS debug server build start capability
Experimental Feature
4New Feature
22- Add Custom Key Mode for Replica Shard Splitting in ClickHouse
- An option for displaying partial results on query cancelation
- Added Support for Arbitrary Table Engines in Temporary Tables
- Add Support for Replication of User-Defined SQL Functions in Keeper
- Implement system.server_settings for Server Configurations
- Support for UNDROP TABLE Query
- Allow Separate Grants for Named Collections in ClickHouse
- Allow Nested Custom Disks in ClickHouse
- Introduce widthBucket function with WIDTH_BUCKET alias
- Add parseDateTime and parseDateTimeInJodaSyntax functions for DateTime parsing
- Use dummy UInt8 for default structure of table function null
- Support for Comma-Separated Date Format in parseDateTimeBestEffort Function
- Add HTTP Settings for Query Wait and Response Buffer Size
- Add system.dropped_tables for tracking incompletely removed dropped tables in Atomic databases
- Add INSTR as alias of positionCaseInsensitive for MySQL compatibility
- Added toDecimalString Function for Fixed Precision Number Conversion
- Add max_number_of_mutations_for_replica Setting to Limit Part Mutations per Replica
- Add mapFromArrays function for creating maps from array pairs
- Allow Control of Compression in Parquet/ORC/Arrow Formats
- Add SSL User Certificate Authentication to Native Protocol
- Add OrNull and OrZero variants for parseDateTime and alias str_to_date for MySQL compatibility
- Added REGEXP Operator for MySQL Compatibility
Improvement
47- Enable input_format_json_ignore_unknown_keys_in_named_tuple by default
- Allow Ignoring Errors When Pushing to MATERIALIZED VIEW with New Setting
- Track Distributed Sends File Queue in Memory
- Now X-ClickHouse-Query-Id and X-ClickHouse-Timezone Headers Added to All HTTP Responses
- External Tables in MongoDB: Replica Set URI Support and Read Preference Option
- Improved Projection Analysis with Query Plan Optimization Setting
- Use Parquet format v2 by default with new version control setting
- New Configuration Syntax for Kafka Topics with Periods in Names
- Fix hyperscan pattern repeat heuristics
- Don't Log ZK Node Error for Concurrent Block Creation by Different Replica
- Increase Opened Files Limit in clickhouse-local for High-CPU Servers
- Exceptions Message Improved for Number Parsing Failures
- Added system.backups update for task progress tracking
- Allow Types Conversion in Native Input Format with New Setting
- Allow IPv4 in range function to generate IP ranges
- Improve Exception Message for Part Movement Between Volumes/Disk
- Support Bool Type in JSONType Function
- Use _request_body Parameter for Predefined HTTP Queries
- Automatic Indentation for SQL Editor on Enter Key Press
- Self-Extraction with 'sudo' Sets Uid and Gid to Running User
- Update repeat function to accept negative integers, aligning with Spark behavior
- Remove::__1 from Stacktraces and Display std::basic_string as String
- Reimplement Interserver Mode to Prevent Replay Attacks
- Improve Regex Group Recognition and Refine regexp_tree Dictionary
- Keeper Enhancement: New 4LW `clrs` Command to Clean Resources
- Add Optional Arguments to Codecs in ClickHouse Compressor and Fix Errors
- Add Big Int Support to runningDifference Function
- Add Expiration Window for S3 Credentials to Prevent ExpiredToken Errors
- Support Decimals and Date32 in Avro Format
- Do not start server on interrupted conversion from Ordinary to Atomic, improved error message and troubleshooting instructions
- Add 'kind' Column to system.opentelemetry_span_log for SpanKind Values
- Allow Simplified Column Naming for Nested Arrays in Protobuf Format
- Added STRICT Modifier for SYSTEM SYNC REPLICA to Wait for Replication Queue
- Improve Naming of OpenTelemetry Span Logs
- Prevent Long Chains of Aggregate Function Combinators
- Support for Subquery in Parameterized Views
- Fix memory leak in MySQL integration with connection_auto_close enabled
- Improved Error Handling for Decimal Parameters
- Fix exact_rows_before_limit for distributed processing and sorting in queries
- ThreadPools Metrics Introspection by Azat Khuzhin
- Add WriteBufferFromS3 Profile Events
- Add --link and --noninteractive options to ClickHouse install
- Fixed UNKNOWN_TABLE exception for materialized views with unavailable dependent tables
- Fix missing path in encrypted disk configuration
- Support for CTE in Parameterized Views Implementation Update
- Support for Big Integers, Flexible Key Types in Map, and Enhanced DateTime64 Precision
- Allow Skipping Errors for Unknown Enum Values in Row Input Formats
Performance Improvement
14- Marks in Memory Compressed, Reducing Usage by 3-6x
- Backups Performance Enhancement in ClickHouse
- Fixed Excessive Reading in Queries with FINAL
- Setting max_final_threads to Improve Concurrency on High-CPU Servers
- Allow Multithreaded Execution of Reading Pipeline for DIRECT Dictionary in ClickHouse
- Optimize Nullable Key Aggregate Performance
- Implemented Lowercase TokenBF_v1 Index Utilization for Token Functions
- Optimize SIMD Usage in position and LIKE Functions
- Optimize Queries for Large Detached Parts in ClickHouse
- Increase max_replicated_merges_in_queue default from 16 to 1000 in ReplicatedMergeTree tables
- Updated clickhouse-copier to enhance partition selection speed using GROUP BY
- Fix ASOF JOIN Performance Degradation
- Even More Batching in Keeper for Improved Read Performance
- Allow PREWHERE for Merge with Different DEFAULT Expressions for Columns
upgrade-notes
7- Lightweight DELETEs Enabled by Default for MergeTree Tables
- Behavior Changes in *domain*RFC and netloc Functions for URL Authority Compliance
- Prohibit Creating Tables with KafkaEngine and Certain Column Statements
- Asynchronous Connection Drain Feature Removed
- Support for 256-bit Decimal Data Type in Various Functions and Protocols
- Make backup/restore threads server settings
- Do Not Allow Const and Non-Deterministic Secondary Indices