v.19.14
Backward Incompatible Change
2Bug Fix
60- This Release Also Contains All Bug Fixes from 19.13 and 19.11.
- Fix Segmentation Fault When the Table Has Skip Indices and Vertical Merge Happens
- Fix Per-column TTL with Non-trivial Column Defaults
- Fix Kafka Messages Duplication Problem on Normal Server Restart
- Fixed Infinite Loop When Reading Kafka Messages
- Fix Key Expression Contains Comparison Between Inconvertible Types Exception in Bitmapcontains Function
- Fix Segfault with Enabled Optimize_skip_unused_shards and Missing Sharding Key
- Fixed Wrong Code in Mutations That May Lead to Memory Corruption
- Removed Extra Verbose Logging in Mysql Interface #6389 (alexey-milovidov)
- Return the Ability to Parse Boolean Settings from 'true' and 'false' in the Configuration File
- Fix Crash in Quantile and Median Function over Nullable(decimal128)
- Fixed Possible Incomplete Result Returned by SELECT Query with WHERE Condition on Primary Key Contained Conversion to Float Type
- Check Max_expanded_ast_elements Setting for Mutations
- Fix JOIN Results for Key Columns When Used with Join_use_nulls
- Fix for Skip Indices with Vertical Merge and Alter
- Fix Rare Crash in ALTER MODIFY COLUMN and Vertical Merge When One of Merged/altered Parts Is Empty (0 Rows) #6746 #6780 (alesapin)
- Fixed Bug in Conversion of Lowcardinality Types in Aggregatefunctionfactory
- Fix Wrong Behavior and Possible Segfaults in Topk and Topkweighted Aggregated Functions
- Fixed Unsafe Code Around Getidentifier Function
- Fixed Bug in Mysql Wire Protocol (is Used While Connecting to Clickhouse Form Mysql Client)
- Fixed Memory Leak in Bitmapsubsetinrange Function
- Fix Rare Bug When Mutation Executed After Granularity Change
- Allow Protobuf Message with All Fields by Default
- Resolve a Bug with Nullif Function When We Send a NULL Argument on the Second Argument
- Fix Rare Bug with Wrong Memory Allocation/deallocation in Complex Key Cache Dictionaries with String Fields Which Leads to Infinite Memory Consumption (looks Like Memory Leak)
- Fixed Gorilla Encoding on Small Sequences Which Caused Exception Cannot Write After End of Buffer
- Allow to Use Not Nullable Types in Joins with Join_use_nulls Enabled
- Disable Poco::abstractconfiguration Substitutions in Query in Clickhouse-client
- Avoid Deadlock in REPLACE PARTITION
- Using Arrayreduce for Constant Arguments May Lead to Segfault
- Fix Inconsistent Parts Which Can Appear If Replica Was Restored After DROP PARTITION
- Fixed Hang in Jsonextractraw Function
- Fix Bug with Incorrect Skip Indices Serialization and Aggregation with Adaptive Granularity
- Fix WITH ROLLUP and WITH CUBE Modifiers of GROUP BY with Two-level Aggregation
- Fix Bug with Writing Secondary Indices Marks with Adaptive Granularity
- Fix Initialization Order While Server Startup
- Clearing the Data Buffer from the Previous Read Operation That Was Completed with an Error
- Fix Bug with Enabling Adaptive Granularity When Creating a New Replica for Replicated*mergetree Table
- Fixed Possible Crash During Server Startup in Case of Exception Happened in Libunwind During Exception at Access to Uninitialized Threadstatus Structure
- Fix Crash in Yandexconsistenthash Function
- Fixed the Possibility of Hanging Queries When Server Is Overloaded and Global Thread Pool Becomes Near Full
- Fixed Logic of Arrayenumerateuniqranked Function
- Fix Segfault When Decoding Symbol Table
- Fixed Irrelevant Exception in Cast of Lowcardinality(nullable) to Not-nullable Column in Case If It Does Not Contain Nulls (e.g
- Removed Extra Quoting of Description in System.settings Table
- Avoid Possible Deadlock in TRUNCATE of Replicated Table
- Fix Reading in Order of Sorting Key
- Fix ALTER TABLE ..
- Fix Bug Opened by #4405 (since 19.4.0)
- Fixed Overflow in Integer Division of Signed Type to Unsigned Type
- Limit Maximum Sleep Time for Throttling When Max_execution_speed or Max_execution_speed_bytes Is Set
- Fixed Issues About Using MATERIALIZED Columns and Aliases in Materializedview
- Fix Formatfactory Behaviour for Input Streams Which Are Not Implemented as Processor
- Fixed Typo
- Typo in the Error Message ( Is -> Are )
- Fixed Error While Parsing of Columns List from String If Type Contained a Comma (this Issue Was Relevant for File, Url, HDFS Storages) #6217
- Fix for Function Аrrayenumerateuniqranked with Empty Arrays in Params
- Fixed Subquery Name in Queries with ARRAY JOIN and GLOBAL IN Subquery with Alias
- This Release Also Contains All Bug Fixes from 19.11.12.69.
- Fixed Compatibility for Distributed Queries Between 19.14 and Earlier Versions
Build/Testing/Packaging Improvement
77- Remove Compiler (runtime Template Instantiation) Because We've Win over It's Performance
- Added Performance Test to Show Degradation of Performance in Gcc-9 in More Isolated Way
- Added Table Function Numbers_mt, Which Is Multi-threaded Version of Numbers
- Comparison Mode in Clickhouse-benchmark #6220 #6343 (dimarub2000)
- Best Effort for Printing Stack Traces
- Every Function in Its Own File, Part 10
- Remove Doubled Const Table_is_read_only
- Formatting Changes for Stringhashmap PR #5417
- Better Subquery for Join Creation in Expressionanalyzer
- Remove a Redundant Condition (found by PVS Studio)
- Separate the Hash Table Interface for Reverseindex
- Refactoring of Settings
- Add Comments for Set Index Functions
- Increase OOM Score in Debug Version on Linux
- HDFS HA Now Work in Debug Build
- Added a Test to Transform_query_for_external_database
- Add Test for Multiple Materialized Views for Kafka Table
- Make a Better Build Scheme
- Fixed Test_external_dictionaries Integration in Case It Was Executed Under Non Root User
- The Bug Reproduces When Total Size of Written Packets Exceeds Dbms_default_buffer_size
- Added a Test for RENAME Table Race Condition #6752 (alexey-milovidov)
- Avoid Data Race on Settings in KILL QUERY
- Add Integration Test for Handling Errors by a Cache Dictionary
- Disable Parsing of ELF Object Files on Mac Os, Because It Makes No Sense
- Attempt to Make Changelog Generator Better
- Adding -wshadow Switch to the GCC
- Removed Obsolete Code for Mimalloc Support
- Zlib-ng Determines X86 Capabilities and Saves This Info to Global Variables
- Regression Test for a Bug Which in Join Which Was Fixed in #5192
- Fixed Msan Report
- Fix Flapping TTL Test
- Fixed False Data Race in Mergetreedatapart::is_frozen Field
- Fixed Timeouts in Fuzz Test
- Added Debug Checks to Static_cast of Columns
- Support for Oracle Linux in Official RPM Packages
- Changed Json Perftests from Once to Loop Type
- Odbc-bridge.cpp Defines Main() So It Should Not Be Included in Clickhouse-lib
- Test for Crash in Full|right JOIN with Nulls in Right Table's Keys
- Added a Test for the Limit on Expansion of Aliases Just in Case
- Switched from Boost::filesystem to Std::filesystem Where Appropriate
- Added RPM Packages to Website
- Add a Test for Fixed Unknown Identifier Exception in IN Section
- Simplify Shared_ptr_helper Because People Facing Difficulties Understanding It
- Added Performance Tests for Fixed Gorilla and Doubledelta Codec
- Split the Integration Test Test_dictionaries into 4 Separate Tests
- Fix Pvs-studio Warning in Pipelineexecutor
- Allow to Use Library Dictionary Source with Asan
- Added Option to Generate Changelog from a List of Prs
- Lock the Tinylog Storage When Reading
- Check for Broken Symlinks in CI
- Increase Timeout for "stack Overflow" Test Because It May Take a Long Time in Debug Build
- Added a Check for Double Whitespaces
- Fix New/delete Memory Tracking When Build with Sanitizers
- Enable Back the Check of Undefined Symbols While Linking
- Avoid Rebuilding Hyperscan Every Day
- Fixed Ubsan Report in Protobufwriter
- Don't Allow to Use Query Profiler with Sanitizers Because It Is Not Compatible
- Add Test for Reloading a Dictionary After Fail by Timer
- Fix Inconsistency in Pipelineexecutor::prepareprocessor Argument Type
- Added a Test for Bad Uris
- Added More Checks to CAST Function
- Added Gcc-9 Support to Docker/builder Container That Builds Image Locally
- Test for Primary Key with Lowcardinality(string)
- Fixed Tests Affected by Slow Stack Traces Printing
- Add a Test Case for Crash in Groupuniqarray Fixed in #6029
- Fixed Indices Mutations Tests
- In Performance Test, Do Not Read Query Log for Queries We Didn't Run
- Materialized View Now Could Be Created with Any Low Cardinality Types Regardless to the Setting About Suspicious Low Cardinality Types
- Updated Tests for Send_logs_level Setting
- Fix Build Under Gcc-8.2
- Fix Build with Internal Libc++
- Fix Shared Build with Rdkafka Library #6101 (ivan)
- Fixes for Mac OS Build (incomplete)
- Fix "splitted" Build
- Other Build Fixes: #6186 (amos Bird) #6486 #6348 (vxider) #6744 (ivan) #6016 #6421 #6491 (proller)
- Fix Flapping.html) Test 00715_fetch_merged_or_mutated_part_zookeeper by Rewriting It to a Shell Scripts Because It Needs to Wait for Mutations to Apply
- Fixed Ubsan and Memsan Failure in Function Groupuniqarray with Emtpy Array Argument
Experimental Feature
2Improvement
59- Correct Implementation of Ternary Logic for And/or
- Now Values and Rows with Expired TTL Will Be Removed After OPTIMIZE ..
- Possibility to Change the Location of Clickhouse History File for Client Using Clickhouse_history_file Env
- Remove Dry_run Flag from Interpreterselectquery
- Support ASOF JOIN with ON Section
- Better Support of Skip Indexes for Mutations and Replication
- Allow to ATTACH Live Views (for Example, at the Server Startup) Regardless to Allow_experimental_live_view Setting
- For Stack Traces Gathered by Query Profiler, Do Not Include Stack Frames Generated by the Query Profiler Itself
- Now Table Functions Values, File, Url, Hdfs Have Support for ALIAS Columns
- Throw an Exception If Config.d File Does Not Have the Corresponding Root Element as the Config File
- Print Extra Info in Exception Message for No Space Left on Device
- When Determining Shards of a Distributed Table to Be Covered by a Read Query (for Optimize_skip_unused_shards = 1) Clickhouse Now Checks Conditions from Both Prewhere and Where Clauses of Select Statement
- Enabled SIMDJSON for Machines Without AVX2 But with SSE 4.2 and PCLMUL Instruction Set
- Clickhouse Can Work on Filesystems Without O_direct Support (such as ZFS and Btrfs) Without Additional Tuning
- Support Push Down Predicate for Final Subquery
- Better JOIN ON Keys Extraction #6131 (artem Zuikov)
- Upated SIMDJSON
- Optimize Selecting of Smallest Column for SELECT Count() Query
- Added Strict Parameter in Windowfunnel()
- Safer Interface of Mysqlxx::pool
- Options Line Size When Executing with --help Option Now Corresponds with Terminal Size
- Disable "read in Order" Optimization for Aggregation Without Keys
- HTTP Status Code for Incorrect_data and Type_mismatch Error Codes Was Changed from Default 500 Internal Server Error to 400 Bad Request
- Move Join Object from Expressionaction into Analyzedjoin
- Fixed Possible Deadlock of Distributed Queries When One of Shards Is Localhost But the Query Is Sent via Network Connection
- Changed Semantic of Multiple Tables RENAME to Avoid Possible Deadlocks
- Rewritten Mysql Compatibility Server to Prevent Loading Full Packet Payload in Memory
- Move AST Alias Interpreting Logic Out of Parser That Does Not Have to Know Anything About Query Semantics
- Slightly More Safe Parsing of Namesandtypeslist
- Clickhouse-copier: Allow Use Where_condition from Config with Partition_key Alias in Query for Checking Partition Existence (earlier It Was Used Only in Reading Data Queries)
- Added Optional Message Argument in Throwif
- Server Exception Got While Sending Insertion Data Is Now Being Processed in Client as Well
- Added a Metric Distributedfilestoinsert That Shows the Total Number of Files in Filesystem That Are Selected to Send to Remote Servers by Distributed Tables
- Move Most of Joins Prepare Logic from Expressionaction/expressionanalyzer to Analyzedjoin
- Fix Tsan Warning.html) 'lock-order-inversion'
- Better Information Messages About Lack of Linux Capabilities
- When Enable Dumping Temporary Data to the Disk to Restrict Memory Usage During GROUP By, ORDER By, It Didn't Check the Free Disk Space
- Removed Recursive Rwlock by Thread
- Split Expressionanalyzer.appendjoin()
- Added Mysql_native_password Authentication Plugin to Mysql Compatibility Server
- Less Number of Clock_gettime Calls; Fixed ABI Compatibility Between Debug/release in Allocator (insignificant Issue)
- Move Collectusedcolumns from Expressionanalyzer to Syntaxanalyzer
- Add Setting Joined_subquery_requires_alias to Require Aliases for Subselects and Table Functions in FROM That More Than One Table Is Present (i.e
- Extract Getaggregatesvisitor Class from Expressionanalyzer
- System.query_log: Change Data Type of Type Column to Enum
- Static Linking of Sha256_password Authentication Plugin
- Avoid Extra Dependency for the Setting Compile to Work
- More Validation of the Input That May Come from Malicious Replica
- Now Clickhouse-obfuscator File Is Available in Clickhouse-client Package
- Fixed Deadlock When We Have at Least Two Queries That Read at Least Two Tables in Different Order and Another Query That Performs DDL Operation on One of Tables
- Added Os_thread_ids Column to System.processes and System.query_log for Better Debugging Possibilities
- A Workaround for PHP Mysqlnd Extension Bugs Which Occur When Sha256_password Is Used as a Default Authentication Plugin
- Remove Unneeded Place with Changed Nullability Columns
- Set Default Value of Queue_max_wait_ms to Zero, Because Current Value (five Seconds) Makes No Sense
- Extract Selectqueryexpressionanalyzer from Expressionanalyzer
- Removed Duplicating Input and Output Formats
- Allow User to Override Poll_interval and Idle_connection_timeout Settings on Connection
- Mergetree Now Has an Additional Option Ttl_only_drop_parts (disabled by Default) to Avoid Partial Pruning of Parts, So That They Dropped Completely When All the Rows in a Part Are Expired
- Type Checks for Set Index Functions
New Feature
24- WITH FILL Modifier for ORDER BY
- WITH TIES Modifier for LIMIT
- Parse Unquoted NULL Literal as NULL (if Setting Format_csv_unquoted_null_literal_as_null=1)
- Support for Wildcards in Paths of Table Functions File and Hdfs
- New System.metric_log Table Which Stores Values of System.events and System.metrics with Specified Time Interval
- Allow to Write Clickhouse Text Logs to System.text_log Table
- Show Private Symbols in Stack Traces (this Is Done via Parsing Symbol Tables of ELF Files)
- Table Function Values (the Name Is Case-insensitive)
- Added an Ability to Alter Storage Settings
- Support for Removing of Detached Parts
- Table Constraints
- Suppport for Cascaded Materialized Views
- Turn on Query Profiler by Default to Sample Every Query Execution Thread Once a Second
- Input Format ORC
- Added Two New Functions: Sigmoid and Tanh (that Are Useful for Machine Learning Applications)
- Function Hastoken(haystack, Token), Hastokencaseinsensitive(haystack, Token) to Check If Given Token Is in Haystack
- New Function Neighbor(value, Offset[, Default_value])
- Created a Function Currentuser(), Returning Login of Authorized User
- New Aggregate Functions Quantilesexactinclusive and Quantilesexactexclusive Which Were Proposed in #5885
- Function Bitmaprange(bitmap, Range_begin, Range_end) Which Returns New Set with Specified Range (not Include the Range_end)
- Function Geohashesinbox(longitude_min, Latitude_min, Longitude_max, Latitude_max, Precision) Which Creates Array of Precision-long Strings of Geohash-boxes Covering Provided Area
- Implement Support for INSERT Query with Kafka Tables
- Added Support for _partition and _timestamp Virtual Columns to Kafka Engine
- Possibility to Remove Sensitive Data from Query_log, Server Logs, Process List with Regexp-based Rules
Performance Improvement
8- Optimize Queries with ORDER BY Expressions Clause, Where Expressions Have Coinciding Prefix with Sorting Key in Mergetree Tables
- Allow to Use Multiple Threads During Parts Loading and Removal
- Implemented Batch Variant of Updating Aggregate Function States
- Using Fastops Library for Functions Exp, Log, Sigmoid, Tanh
- Disable Consecutive Key Optimization for Uint8/16
- Improved Performance of Simdjson Library by Getting Rid of Dynamic Allocation in Parsedjson::iterator
- Pre-fault Pages When Allocating Memory with Mmap()
- Fix Performance Bug in Decimal Comparison