VMware Cloud Community
jl0815
Contributor
Contributor

different disk sizes per host

Hi,

we have a 8 ESXi cluster where 4 ESXi providing vSAN at the moment. Each of these host has a 400 GB SSD + 5x 900 GB SAS disks.

Now we want to extend the vSAN capacity by putting new SSDs and disks to the other 4 hosts.

Is there any problem if we use 400 GB SSD + 5x 1.8 TB disks instead of 900 GB disks?

Thanks,

JL

Reply
0 Kudos
6 Replies
TheBobkin
Champion
Champion

Hello,

It is advised to have homogeneous disk-groups through-out the cluster.

While this is feasible (and supported provided the drives on are on vSAN HCL), this type of configuration may not perform as well as expected as the cluster may end up 'waiting' on the slower disk-groups and thus the overall access-speed of the cluster may be reduced (or at least not improved as much as would be expected when doubling the number of disks/disk-groups). Will these disk-groups be going through the single existing controller? - This may also limit performance.

Also consider the impact in failure scenarios, e.g. if the larger capacity disk-group failed (due to SSD failure) the resultant rebuild might fill the remaining smaller disk-group (not so likely on an 8-node cluster unless running at very high % space utilisation).

In these new disk-groups - 4.4% cache:capacity per disk-group also falls far short of the recommended 10% (of used) ratio.

So in summary - Yes this can be done but there may be impacts to consider as noted above.

Bob

Reply
0 Kudos
RAJ_RAJ
Expert
Expert

HI ,

You add new host with the capacity of disk you have mentioned .

Each disk on the vSAN will have its own file system, and the full capacity can be used by VSAN .

RAJESH RADHAKRISHNAN VCA -DCV/WM/Cloud,VCP 5 - DCV/DT/CLOUD, ,VCP6-DCV, EMCISA,EMCSA,MCTS,MCPS,BCFA https://ae.linkedin.com/in/rajesh-radhakrishnan-76269335 Mark my post as "helpful" or "correct" if I've helped resolve or answered your query!
Reply
0 Kudos
RAJ_RAJ
Expert
Expert

Hi ,

You can use mixed hard drives with different capacity on nodes , but always better keep same to get full performance .

It is not a big challenge since all writes and majority of reads will be handled by FLASH  , still there will be small variation on performance .

Each disk on the vSAN will have its own file system, and the full capacity can be used by VSAN  .

RAJESH RADHAKRISHNAN VCA -DCV/WM/Cloud,VCP 5 - DCV/DT/CLOUD, ,VCP6-DCV, EMCISA,EMCSA,MCTS,MCPS,BCFA https://ae.linkedin.com/in/rajesh-radhakrishnan-76269335 Mark my post as "helpful" or "correct" if I've helped resolve or answered your query!
Reply
0 Kudos
TheBobkin
Champion
Champion

Hello Rajesh,

"It is not a big challenge since all writes and majority of reads will be handled by FLASH"

Sorry, but this is false and thus confusing - writes have to be committed to capacity-tier at some point so saying "all writes" are handled by flash doesn't make sense as this is only used as an intermediate buffer.

While it is possible that blocks can get changed and put back in buffer before hitting capacity-tier this is unlikely to be the bulk of writes unless there was a really high usable cache:capacity tier ratio and/or a workload that frequently changes the same data.

Bob

Reply
0 Kudos
RAJ_RAJ
Expert
Expert

HI ,

if  the word Writes handles misleading just to clarify - it doesn't mean that not writing data to SSD permanently .  As you mentioned SSD will be in cache tier

and it really helps I/O on vSAN .

About Read and Write on SSD is explained detail on below .

VSAN Part 8 - The role of the SSD - CormacHogan.com

RAJESH RADHAKRISHNAN VCA -DCV/WM/Cloud,VCP 5 - DCV/DT/CLOUD, ,VCP6-DCV, EMCISA,EMCSA,MCTS,MCPS,BCFA https://ae.linkedin.com/in/rajesh-radhakrishnan-76269335 Mark my post as "helpful" or "correct" if I've helped resolve or answered your query!
Reply
0 Kudos
memmaker
Contributor
Contributor

I think Bobkin and RAJ, both of you are talking about same thing.. Smiley Happy

Yes. it's cache tier, you guys may know what cache is doing.

VMware recommend to use same size of cache and capacity disk. but it's not big problem when you add different capacity tier disks. it just may cause some imbalances. but it's not big deal.

but, for cache disk sizing? you may need to know how many space your VM will be actually using without FTT consideration.

for example, you have 3 VMs which has 1TB per each with FTT=1, RAID-1

so roughly 6 TB will be consumed. however, for cache sizing, FTT is not considered.

so 10% of 3TB = 300GB is your cache disk size requirement.

for AF there has been change..

Designing vSAN Disk groups - All Flash Cache Ratio Update - Virtual Blocks

Reply
0 Kudos