v.19.16
New Feature
14- Add Deduplicate_blocks_in_dependent_materialized_views Option to Control the Behaviour of Idempotent Inserts into Tables with Materialized Views
- Introduce Uniqcombined64() to Calculate Cardinality Greater Than Uint_max
- Support Bloom Filter Indexes on Array Columns
- Add a Function Getmacro(name) That Returns String with the Value of Corresponding <macros> from Server Configuration
- Set Two Configuration Options for a Dictionary Based on an HTTP Source: Credentials and Http-headers
- Add a New Profileevent Merge That Counts the Number of Launched Background Merges
- Add Fullhostname Function That Returns a Fully Qualified Domain Name
- Add Function Arraysplit and Arrayreversesplit Which Split an Array by "cut Off" Conditions
- Add New Functions That Return the Array of All Matched Indices in Multimatch Family of Functions
- Add a New Database Engine Lazy That Is Optimized for Storing a Large Number of Small -log Tables
- Add Aggregate Functions Groupbitmapand, -or, -xor for Bitmap Columns
- Add Aggregate Function Combinators -ornull and -ordefault, Which Return Null or Default Values When There Is Nothing to Aggregate
- Introduce Customseparated Data Format That Supports Custom Escaping and Delimiter Rules
- Support Redis as Source of External Dictionary
Backward Incompatible Change
3Bug Fix
13- Fix Wrong Query Result If It Has WHERE IN (select ...) Section and Optimize_read_in_order Is Used
- Disabled Mariadb Authentication Plugin, Which Depends on Files Outside of Project
- Fix Exception Cannot Convert Column ..
- Fixed Issue of Using HTTP Keep Alive Timeout Instead of TCP Keep Alive Timeout
- Fixed a Segmentation Fault in Groupbitmapor
- For Materialized Views the Commit for Kafka Is Called After All Data Were Written
- Fixed Wrong Duration_ms Value in System.part_log Table
- A Quick Fix to Resolve Crash in LIVE VIEW Table and Re-enabling All LIVE VIEW Tests
- Serialize NULL Values Correctly in Min/max Indexes of Mergetree Parts
- Don't Put Virtual Columns to .sql Metadata When Table Is Created as CREATE TABLE AS
- Fix Segmentation Fault in ATTACH PART Query
- Fix Wrong Result for Some Queries Given by the Optimization of Empty IN Subqueries and Empty Inner/right JOIN
- Fixing Addresssanitizer Error in the LIVE VIEW Getheader() Method
Build/Testing/Packaging Improvement
23- Disable Some Contribs for Cross-compilation to Mac OS
- Add Missing Linking with Pocoxml for Clickhouse_common_io
- Accept Multiple Test Filter Arguments in Clickhouse-test
- Enable Musl and Jemalloc for ARM
- Added --client-option Parameter to Clickhouse-test to Pass Additional Parameters to Client
- Preserve Existing Configs on Rpm Package Upgrade
- Fix Errors Detected by PVS
- Fix Build for Darwin
- Glibc 2.29 Compatibility
- Make Sure Dh_clean Does Not Touch Potential Source Files
- Attempt to Avoid Conflict When Updating from Altinity Rpm - It Has Config File Packaged Separately in Clickhouse-server-common
- Optimize Some Header Files for Faster Rebuilds
- Add Performance Tests for Date and Datetime
- Fix Some Tests That Contained Non-deterministic Mutations
- Add Build with Memorysanitizer to CI
- Avoid Use of Uninitialized Values in Metricstransmitter
- Fix Some Issues in Fields Found by Memorysanitizer
- Fix Undefined Behavior in Murmurhash32
- Fix Undefined Behavior in Storagesinfostream
- Fixed Constant Expressions Folding for External Database Engines (mysql, Odbc, Jdbc)
- Fixing Threadsanitizer Data Race Error in the LIVE VIEW When Accessing No_users_thread Variable
- Get Rid of Malloc Symbols in Libcommon #7134, #7065 (amos Bird)
- Add Global Flag Enable_libraries for Disabling All Libraries
Code Cleanup
11- Generalize Configuration Repository to Prepare for DDL for Dictionaries
- Parser for Dictionaries DDL Without Any Semantic
- Split Parsercreatequery into Different Smaller Parsers
- Small Refactoring and Renaming Near External Dictionaries
- Refactor Some Code to Prepare for Role-based Access Control
- Some Improvements in Databaseordinary Code
- Do Not Use Iterators in Find() and Emplace() Methods of Hash Tables
- Fix Getmultiplevaluesfromconfig in Case When Parameter Root Is Not Empty
- Remove Some Copy-paste (temporaryfile and Temporaryfilestream) #7166 (artem Zuikov)
- Improved Code Readability a Little Bit (mergetreedata::getactivecontainingpart)
- Wait for All Scheduled Jobs, Which Are Using Local Objects, If Threadpool::schedule(...) Throws an Exception
Improvement
12- Add a Message in Case of Queue_wait_max_ms Wait Takes Place
- Made Setting S3_min_upload_part_size Table-level
- Check TTL in Storagefactory
- Squash Left-hand Blocks in Partial Merge Join (optimization)
- Do Not Allow Non-deterministic Functions in Mutations of Replicated Table Engines, Because This Can Introduce Inconsistencies Between Replicas
- Disable Memory Tracker While Converting Exception Stack Trace to String
- Miscellaneous Format Improvements
- Clickhouse Ignores Values on the Right Side of IN Operator That Are Not Convertible to the Left Side Type
- Support Missing Inequalities for ASOF JOIN
- Optimize Partial Merge Join
- Do Not Use More Than 98K of Memory in Uniqcombined Functions
- Flush Parts of Right-hand Joining Table on Disk in Partialmergejoin (if There Is Not Enough Memory)