v.19.11
Bug Fix
72- Do Store Offsets for Kafka Messages Manually to Be Able to Commit Them All at Once for All Partitions
- Fix Logical Error Causing Segfaults When Selecting from Kafka Empty Topic
- Fix for Function АrrayEnumerateUniqRanked with Empty Arrays in Params
- Fixed Performance Degradation of Index Analysis on Complex Keys on Large Tables
- Avoid Rare SIGSEGV While Sending Data in Tables with Distributed Engine (failed to Send Batch: File with Index XXXXX Is Absent)
- Fix Unknown Identifier with Multiple Joins
- Fixed Rare Crash in ALTER MODIFY COLUMN and Vertical Merge When One of Merged/altered Parts Is Empty (0 Rows)
- Manual Update of SIMDJSON
- Fixed Bug with Mrk File Extension for Mutations (alesapin)
- Implement DNS Cache with Asynchronous Update
- Fix Segfault in Delta Codec Which Affects Columns with Values Less Than 32 Bits Size
- Fix Segfault in TTL Merge with Non-physical Columns in Block
- Fix Rare Bug in Checking of Part with Lowcardinality Column
- Avoid Hanging Connections When Server Thread Pool Is Full
- Support for Constant Arguments to Evalmlmodel Function
- Fixed the Issue When Clickhouse Determines Default Time Zone as UCT Instead of UTC
- Fixed Buffer Underflow in Visitparamextractraw
- Now Distributed Drop/alter/truncate/optimize ON CLUSTER Queries Will Be Executed Directly on Leader Replica
- Fix Coalesce for Columnconst with Columnnullable + Related Changes
- Fix the Readbufferfromkafkaconsumer So That It Keeps Reading New Messages After Commit() Even If It Was Stalled Before #5852 (ivan)
- Fix FULL and RIGHT JOIN Results When Joining on Nullable Keys in Right Table
- Possible Fix of Infinite Sleeping of Low-priority Queries
- Fix Race Condition, Which Cause That Some Queries May Not Appear in Query_log After SYSTEM FLUSH LOGS Query
- Fixed Heap-use-after-free Asan Warning in Clustercopier Caused by Watch Which Try to Use Already Removed Copier Object
- Fixed Wrong Stringref Pointer Returned by Some Implementations of Icolumn::deserializeandinsertfromarena
- Prevent Source and Intermediate Array Join Columns of Masking Same Name Columns
- Fix Insert and Select Query to Mysql Engine with Mysql Style Identifier Quoting
- Now CHECK TABLE Query Can Work with Mergetree Engine Family
- Fix Split_shared_libraries Runtime #5793 (danila Kutenin)
- Fixed Time Zone Initialization When /etc/localtime Is a Relative Symlink Like ../usr/share/zoneinfo/asia/istanbul #5922 (alexey-milovidov)
- Clickhouse-copier: Fix Use-after Free on Shutdown #5752 (proller)
- Updated Simdjson
- Fix Shutdown of Systemlogs #5802 (anton Popov)
- Fix Hanging When Condition in Invalidate_query Depends on a Dictionary
- Fix Bug with Writing Secondary Indices Marks with Adaptive Granularity
- Fix WITH ROLLUP and WITH CUBE Modifiers of GROUP BY with Two-level Aggregation
- Fixed Hang in Jsonextractraw Function
- Fix Segfault in Externalloader::reloadoutdated()
- Fixed the Case When Server May Close Listening Sockets But Not Shutdown and Continue Serving Remaining Queries
- Fixed Useless and Incorrect Condition on Update Field for Initial Loading of External Dictionaries via Odbc, Mysql, Clickhouse and HTTP
- Fixed Irrelevant Exception in Cast of Lowcardinality(nullable) to Not-nullable Column in Case If It Does Not Contain Nulls (e.g
- Fix Non-deterministic Result of "uniq" Aggregate Function in Extreme Rare Cases
- Segfault When We Set a Little Bit Too High CIDR on the Function Ipv6cidrtorange
- Fixed Small Memory Leak When Server Throw Many Exceptions from Many Different Contexts
- Fix the Situation When Consumer Got Paused Before Subscription and Not Resumed Afterwards
- Clearing the Kafka Data Buffer from the Previous Read Operation That Was Completed with an Error #6026 (nikolay) Note That Kafka Is Broken in This Version.
- Since Storagemergetree::background_task_handle Is Initialized in Startup() the Mergetreeblockoutputstream::write() May Try to Use It Before Initialization
- Fixed the Possibility of Hanging Queries When Server Is Overloaded
- Fix FPE in Yandexconsistenthash Function
- Fixed Bug in Conversion of Lowcardinality Types in Aggregatefunctionfactory
- Fix Parsing of Bool Settings from True and False Strings in Configuration Files
- Fix Rare Bug with Incompatible Stream Headers in Queries to Distributed Table over Mergetree Table When Part of WHERE Moves to PREWHERE
- Fixed Overflow in Integer Division of Signed Type to Unsigned Type
- Kafka Integration Has Been Fixed in This Version.
- Fix Segfault When Using Arrayreduce for Constant Arguments
- Fixed Tofloat() Monotonicity
- Fix Segfault with Enabled Optimize_skip_unused_shards and Missing Sharding Key
- Fixed Logic of Arrayenumerateuniqranked Function
- Removed Extra Verbose Logging from Mysql Handler
- Fix Wrong Behavior and Possible Segfaults in Topk and Topkweighted Aggregated Functions
- Do Not Expose Virtual Columns in System.columns Table
- Fix Bug with Memory Allocation for String Fields in Complex Key Cache Dictionary
- Fix Bug with Enabling Adaptive Granularity When Creating New Replica for Replicated*mergetree Table
- Fix Infinite Loop When Reading Kafka Messages
- Fixed the Possibility of a Fabricated Query to Cause Server Crash Due to Stack Overflow in SQL Parser and Possibility of Stack Overflow in Merge and Distributed Tables #6433 (alexey-milovidov)
- Fixed Gorilla Encoding Error on Small Sequences
- Fix ALTER TABLE ..
- Fix NPE When Using IN Clause with a Subquery with a Tuple
- Fixed an Issue That If a Stale Replica Becomes Alive, It May Still Have Data Parts That Were Removed by DROP PARTITION
- Fixed Issue with Parsing CSV #6426 #6559 (tavplubix)
- Fixed Data Race in System.parts Table and ALTER Query
- Fixed Wrong Code in Mutations That May Lead to Memory Corruption
Backward Incompatible Change
5Build/Testing/Packaging Improvement
38- Add a Backward Compatibility Test for Client-server Interaction with Different Versions of Clickhouse
- Test Coverage Information in Every Commit and Pull Request
- Cooperate with Address Sanitizer to Support Our Custom Allocators (arena and Arenawithfreelists) for Better Debugging of "use-after-free" Errors
- Switch to LLVM Libunwind Implementation for C++ Exception Handling and for Stack Traces Printing #4828 (nikita Lapkov)
- Add Two More Warnings from -weverything #5923 (alexey-milovidov)
- Allow to Build Clickhouse with Memory Sanitizer
- Fixed Ubsan Report About Bittest Function in Fuzz Test
- Docker: Added Possibility to Init a Clickhouse Instance Which Requires Authentication
- Update Librdkafka to Version 1.1.0 #5872 (ivan)
- Add Global Timeout for Integration Tests and Disable Some of Them in Tests Code
- Fix Some Threadsanitizer Failures
- The --no-undefined Option Forces the Linker to Check All External Names for Existence While Linking
- Added Performance Test for #5797 #5914 (alexey-milovidov)
- Fixed Compatibility with Gcc-7
- Added Support for Gcc-9
- Fixed Error When Libunwind Can Be Linked Incorrectly
- Fixed a Few Warnings Found by Pvs-studio
- Added Initial Support for Clang-tidy Static Analyzer
- Convert Bsd/linux Endian Macros( 'be64toh' and 'htobe64') to the Mac OS X Equivalents #5785 (fu Chen)
- Improved Integration Tests Guide
- Fixing Build at Macosx + Gcc9 #5822 (filimonov)
- Fix a Hard-to-spot Typo: Aggreagte -> Aggregate
- Fix Freebsd Build #5760 (proller)
- Add Link to Experimental Youtube Channel to Website #5845 (ivan Blinkov)
- Cmake: Add Option for Coverage Flags: With_coverage #5776 (proller)
- Fix Initial Size of Some Inline Podarray's
- Clickhouse-server.postinst: Fix Os Detection for Centos 6 #5788 (proller)
- Added Arch Linux Package Generation
- Split Common/config.h by Libs (dbms) #5715 (proller)
- Fixes for "arcadia" Build Platform #5795 (proller)
- Fixes for Unconventional Build (gcc9, No Submodules) #5792 (proller)
- Require Explicit Type in Unalignedstore Because It Was Proven to Be Bug-prone #5791 (akuzm)
- Fixes Macos Build #5830 (filimonov)
- Performance Test Concerning the New JIT Feature with Bigger Dataset, as Requested Here #5263 #5887 (guillaume Tassery)
- Run Stateful Tests in Stress Test 12693e568722f11e19859742f56428455501fd2a (alesapin)
- Added Official Rpm Packages
- Add an Ability to Build .rpm and .tgz Packages with Packager Script
- Fixes for "arcadia" Build System
Improvement
20- Allow Unresolvable Addresses in Cluster Configuration
- Close Idle TCP Connections (with One Hour Timeout by Default)
- Better Quality of Topk Function
- URL Functions to Work with Domains Now Can Work for Incomplete Urls Without Scheme #5725 (alesapin)
- Checksums Added to the System.parts_columns Table
- Added Enum Data Type as a Synonim for Enum8 or Enum16
- Full Bit Transpose Variant for T64 Codec
- Condition on Startswith Function Now Can Uses Primary Key
- Allow to Use Clickhouse-copier with Cross-replication Cluster Topology by Permitting Empty Database Name
- Use UTC as Default Timezone on a System Without Tzdata (e.g
- Returned Back Support for Floating Point Argument in Function Quantiletiming for Backward Compatibility
- Show Which Table Is Missing Column in Error Messages
- Disallow Run Query with Same Query_id by Various Users #5430 (proller)
- More Robust Code for Sending Metrics to Graphite
- More Informative Error Messages Will Be Displayed When Threadpool Cannot Schedule a Task for Execution
- Inverting Ngramsearch to Be More Intuitive #5807 (danila Kutenin)
- Add User Parsing in HDFS Engine Builder #5946 (akonyaev90)
- Update Default Value of Max_ast_elements Parameter #5933 (artem Konovalov)
- Added a Notion of Obsolete Settings
- Allow User to Override Poll_interval and Idle_connection_timeout Settings on Connection
New Feature
8- Added Support for Prepared Statements
- Doubledelta and Gorilla Column Codecs #5600 (vasily Nemkov)
- Added Os_thread_priority Setting That Allows to Control the "nice" Value of Query Processing Threads That Is Used by OS to Adjust Dynamic Scheduling Priority
- Implement _topic, _offset, _key Columns for Kafka Engine #5382 (ivan) Note That Kafka Is Broken in This Version.
- Add Aggregate Function Combinator -resample #5590 (hcz)
- Aggregate Functions Grouparraymovingsum(win_size)(x) and Grouparraymovingavg(win_size)(x), Which Calculate Moving Sum/avg with or Without Window-size Limitation
- Add Synonim Arrayflatten <-> Flatten #5764 (hcz)
- Intergate H3 Function Geotoh3 from Uber