Support Deduplication for Asynchronous Inserts in ClickHouse

Why it matters

Previously, asynchronous inserts did not support deduplication because multiple small inserts were merged into a single batch, preventing proper duplicate detection. This feature solves that problem by enabling deduplication within asynchronous insert batches, improving data integrity and reducing unwanted duplicates without sacrificing async insert performance.

How to use it

Deduplication in asynchronous inserts is enabled automatically when async insert mode is used with the deduplication setting configured. Users can perform asynchronous inserts as usual with the INSERT statement, and the system will handle deduplication internally for the batched inserts.