v.24.9Improvement

Improve DateTime Schema Inference with Stricter Handling

Official CH changelog text
Improve schema inference of date times. Now DateTime64 used only when date time has fractional part, otherwise regular DateTime is used. Inference of Date/DateTime is more strict now, especially when date_time_input_format='best_effort' to avoid inferring date times from strings in corner cases. #68382 (Kruglov Pavel).
Improved schema inference for date and date-time values, using DateTime64 only when fractional seconds are present, and stricter inference rules for Date and DateTime types, especially with date_time_input_format='best_effort'.

Why it matters

To enhance the accuracy of automatic schema detection for date and date-time columns by avoiding incorrect inferences in edge cases and only using DateTime64 when fractional parts exist. This reduces mistaken data type assignments and improves data handling consistency.

How to use it

Users benefit from improved inference automatically when importing data. To leverage stricter inference, set the setting date_time_input_format='best_effort' when parsing date-time strings to ensure safer and more precise type detection.

Related resources

PreviousNext