Avoid Unnecessary Reconstruction of Index Granules in Skip Indexes

Avoid unnecessary reconstruction of index granules when reading skip indexes. This addresses #55653. #55683 (Amos Bird).