Fix schema inference for invalid format detection

Fix invalid format detection in schema inference that could lead to logical error Format doesn't support schema inference. #66899 (Kruglov Pavel).