v.24.1Improvement

Custom Disk Configuration for Filesystem Caches in ClickHouse

Make sure that for custom (created from SQL) disks ether filesystem_caches_path (a common directory prefix for all filesystem caches) or custom_cached_disks_base_directory (a common directory prefix for only filesystem caches created from custom disks) is specified in server config. custom_cached_disks_base_directory has higher priority for custom disks over filesystem_caches_path, which is used if the former one is absent. Filesystem cache setting path must lie inside that directory, otherwise exception will be thrown preventing disk to be created. This will not affect disks created on an older version and server was upgraded - then the exception will not be thrown to allow the server to successfully start). custom_cached_disks_base_directory is added to default server config as /var/lib/clickhouse/caches/. Closes #57825. #58869 (Kseniia Sumarokova).
Introduced configuration requirements to ensure that custom disks created from SQL specify either filesystem_caches_path or custom_cached_disks_base_directory in the server config to properly define the location for filesystem caches.

Why it matters

This feature ensures that all filesystem caches related to custom disks reside within a designated directory prefix, improving configuration safety and preventing misconfigurations that could lead to disks being created with invalid cache paths. It also prioritizes custom_cached_disks_base_directory for custom disks, providing better isolation of cache directories and reducing the risk of conflicts or errors.

How to use it

To use this feature, specify either filesystem_caches_path or custom_cached_disks_base_directory in your server configuration. For stronger isolation and priority, set custom_cached_disks_base_directory (default is /var/lib/clickhouse/caches/) for your custom disks. Make sure the path of any filesystem cache lies within the configured directory; otherwise, the server will throw an exception preventing disk creation.