how does mixing disk groups all flash and hybrid affect performance?

[justinm001]

If we have VSan with 4 servers and all 4 have the same configuration 

If we have a disk group with each having  1-480GB SSD and 2-1TB 7200,  then we add another disk group with 1-400GB NVMe and 2-1.2TB SSD, how would that affect performance and is there any way we can have more important VM's run on the all flash while keeping the others on Hybrid?  I don't see a way to have separate Vsan datastores, is there one?

Also how would it be affected if we went the other way and added 1-400GB SSD and 4-1TB 5400 drives.  Adding disk groups should always increase IOPS, but what if the drives are slower than others?

[TheBobkin]

Hello Justin,

Welcome to Communities.

"Mixing disk group types (all-flash and hybrid) is not supported.":

storagehub.vmware.com/export_to_pdf/vsan-frequently-asked-questions-faq

How would it function - if it was in anyway stable (unlikely) you would likely not see any increase in performance as the 'fast' disk-groups would end up waiting on the 'slow' disk-groups to perform their correlating part of the writes.

"can have more important VM's run on the all flash while keeping the others on Hybrid"

No, there is no method in standard vSAN cluster of choosing what goes where (the nearest to this would be either locality and/or PFTT=0 in Stretched-cluster).

"I don't see a way to have separate Vsan datastores, is there one?"

Nope, one vsanDatastore per cluster.

While yes, faster or equal drives should always be used for replacements, I would bet on the 4x 5400 HDD vs 2x 7200 HDD for performance (assuming nothing else drastically different e.g. SATA vs SAS) - though this being said, one should always aim for the nearest possible to homogeneity in the configuration of vSAN nodes.

Bob

[justinm001]

Didn't realize that mixing Hybrid and all flash disk groups isn't supported.  What would be the best recommendation for providing 4 servers with the best performance/capacity using the drives below?  If we go all flash, we'll lose on the 8TB, but if we do all mixed will the performance be that much worse? 

4x 480GB SSD

4x 1.2TB SSD

4x 400GB SSD PCIE NVME

8x 1TB 7200

[TheBobkin · Accepted solution]

Hello Justin,

Running many unsupported configurations in vSAN (while technically possible) are a bad idea which can result in anything from mild hiccups to full data-loss, not worth the risk.

The available storage-space *lost* by not using HDDs could be made up for by using RAID 5/6 instead of RAID1 (1.33x vs 2x space used assuming FTT=1).

Also note in case you are using Standard vSAN license:

All-Flash configuration is only licensed with vSAN Advanced/Enterprise in 6.0/6.2:

www.vmware.com/files/pdf/products/vsan/vmware-vsan-62-licensing-guide.pdf

While All-Flash is licensed in vSAN Standard in 6.5, RAID 5/6 is not:

www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/products/vsan/vmware-vsan-65-licensing-gui...

My advice if you do want to go all-flash would be to buy more ~1TB SSDs and use those + the NVMe devices as cache (provided these are certified for All-Flash cache-tier role).

Bob

[justinm001]

We're licensed for Vsan advanced and are using Intel DC P3600 NVMe which are on HCL so we should be good to go. We're going to go RAID5/6 and all flash on this then compare to our other test environment thats hybrid and see the performance differences.