v.21.6Improvement
Add non_replicated_deduplication_window setting for MergeTree inserts
Add new setting non_replicated_deduplication_window for non-replicated MergeTree inserts deduplication. #22514 (alesapin).Why it matters
This feature addresses the need to avoid duplicate data insertions in non-replicated MergeTree tables by providing a configurable deduplication window. It helps ensure data consistency and prevents unintentional duplicate rows without relying on replication mechanisms.How to use it
To use this feature, set thenon_replicated_deduplication_window setting to the desired duration (time interval) before performing inserts on non-replicated MergeTree tables. This activates deduplication for inserts within the specified time window.