v.21.6Improvement

Add non_replicated_deduplication_window setting for MergeTree inserts

Add new setting non_replicated_deduplication_window for non-replicated MergeTree inserts deduplication. #22514 (alesapin).
Added a new setting non_replicated_deduplication_window to enable deduplication of inserts for non-replicated MergeTree tables.

Why it matters

This feature addresses the need to avoid duplicate data insertions in non-replicated MergeTree tables by providing a configurable deduplication window. It helps ensure data consistency and prevents unintentional duplicate rows without relying on replication mechanisms.

How to use it

To use this feature, set the non_replicated_deduplication_window setting to the desired duration (time interval) before performing inserts on non-replicated MergeTree tables. This activates deduplication for inserts within the specified time window.