vSAN NVMe cache and disk group assignment

[max_ymoc]

Hello Community

I have a question about vSAN management.
I came across a vSAN consisting of 4 Hosts, FDTT: 1 and Single Site. Deduplication and Compression are disabled.
In Disk Management I find 2 Disk groups all flash.
The first disk group consists of 3 SSD disks, the capacitive 2 disks with a size of 7TB each and a 4TB cache disk.
The second disk group consists of a 1GB NVMe disk as a cache and the second disk is a capacitive 7TB NVMe.
Customer would like to move 7TB NVMe disk and promote it as the first disk group cache disk.

Correct me if I'm wrong. To do this I have to put one node in maintenance mode at a time and perform a full migration of both disk groups and at the end remove them.
Only then can I reconfigure the disk groups as requested by the Client and proceed with the next node until all nodes are completed.
Is this procedure correct?

[kastlr]

Hi,

is your customer aware that vSAN will only use 600 GB of his 7TB NVMe "Cache" Device?

Understanding vSAN Architecture: Disk Groups

As removal of a Cache SSD will cause a complete DG to be unavailable you have to perform a full data migration for each node.

[TheBobkin]

@max_ymoc I completely agree with what @kastlr said in that using a 7TB device (or even a 4TB device really) is a waste of NAND as vSAN is only going to use 600GB of the device for write-buffer and any space above this will basically only be used to prolong the lifespan of the device via wear-leveling. You likely won't see any benefit from a performance perspective unless the larger device is an overall faster device (e.g. more write-oriented).

Replacing a Cache-tier device means removing and recreating the Disk-Group - if you have all data stored with a RAID5 storage policy here then 'Full Data Migration' option won't really be feasible (unless overall space usage is lower than ~45%), if all data is stored as RAID1 then less free space would be needed to do this. If 'Full Data Migration' isn't viable then ensure you take backups prior to starting changes and use 'Ensure Accessibility' option.