Different size Disk Groups

[lucasbernadsky]

Hi, in our environment we have 1 diskgroup per host. All same capacity (Hybrid configuration). As we are running out of disk space, we want to increase our vSAN datastore capacity by adding a Diskgroup with bigger disks (1 for cache and 7 equal disks for capacity) for each host in the cluster.

Is it recommended to use Disk Groups with different capacity within the same host when using vSAN?

What would be the impact on that? Is it supported by VMware?

Thanks in advanced

[TheBobkin · Accepted solution]

Hello Lucas,

It's supported but it's not recommended - understanding the reasons why it is not recommended will better enable you to plan and weigh your options (e.g. it is big wide grey area to how strongly it would be not recommended).

So, basically it breaks down to performance and capacity: if you have disks or Disk-Groups that are lower performance spec relative to the current smaller DGs (or even just worse relative to their capacity) the performance will likely be decreased. These also are now storing the relative majority of the data (because they are bigger) and thus will be relatively busier, this means more pressure on both capacity and cache-tier and there is a limit to how big a cache-tier is useful so simply scaling up both cache and capacity isn't allows ideal.

Is there a specific reason that you want to expand with disks that are larger than the current ones? You will probably have to pay more for larger cache-tier (to maintain good cache:capacity ratio) negating some of the savings made on buying larger capacity-tier disks - don't just consider price and size while buying HDDs either though, lowest performance SATA is worlds apart from good fast SAS, but then again this comes back to cost. You could also consider shuffling the original smaller drives and the newer larger ones in mixed size Disk-Groups to avoid one being slower than the other but this is not ideal either.

How are you currently using the space in this cluster and how well do you know the current data-set on it?

You may be able to reduce current space usage - I work in GSS and sometimes find customers with large proportions of their used space as *wasted* in a number of ways:

- Snapshots (don't go by Snapshot Manager, check the disk paths or check with # find /vmfs/volumes/datastoreName/ | grep 000).

- Objects that are Thick either intentionally or unintentionally (search on Communities for approach - I have covered this in-depth multiple times).

- Test VMs (there are even ways by which test VMs created by Proactive VM creation test or benchmarking tools remain using space due to testing or encountering issues such as network partitions in old builds).

- VMs that were unregistered, restored or cloned for whatever reason but original is still stored on datastore.

- Thick vswps - if you have good physical memory to allocated VM memory ratio these should be Thin.

- Random detritus...I once saw a 10TB test cluster with 4TB used by the left over data-replica of a (presumably) mistakenly provisioned test VM with 4TB memory assigned and the vswp made inaccessible via fault-testing and never cleaned up.

Bob

---------------------------------------------------------------------------------------------------------

Was it helpful? Let us know by completing this short survey here.

[lucasbernadsky]

Hi, Bob.

Firstly, I want to thank you for taking the time to answer my question with such dedication.

I didn't provide enough information in my question:

With the current scenario, we have 1 Disk Group with 7 300GB 15K SAS drives for capacity, for each of our 4 hosts in the cluster.

The customer has plenty of compute space left, but not as much disk capacity, so we are evaluating posibilites. We 've talked to our hardware provider and they say that the smaller disk they have are 600 GB ones (Also 15K SAS). As we don't want to mix disks within the Disk Group, we simply want to add a new one, so each host would have 1 Disk Group with 7 300 GB disks and 1 DiskGroup with 5 *All-equal larger disks* and its cache.

What do you think?

Thanks again!

[TheBobkin]

Happy to help - I don't think I am capable of writing one-line responses :smileygrin:

That's not as extreme a change as I was imagining (e.g. throwing some 4TB 7.2k SATA in with existing 2TB 15k SAS).

The current configuration is a relatively fast configuration for Hybrid and the performance hit from slower (relative to the amount of data they are carrying) disks isn't necessarily going to be noticeable unless it is already reaching the maximums of its storage performance capabilities - e.g. imagine a road has 100 km/h speed limit but you don't drive more than 60 km/h, it's not going to be noticeable if the speed limit is reduced to 80 km/h. Adding extra DGs can vastly improve the performance of the system so initially it will likely actually be overall faster but this will likely be decreased once the new storage space is more full as it will be handling more IOs. Another thing to bear in mind is failure and rebuild scenarios - e.g. if you lose a larger DG it would have more of an impact than a smaller one (e.g. in a small cluster, not enough space to rebuild on remaining DG and more data to be resynced), that being said current DG isn't massively smaller (2.1TB vs 3TB).

What size Cache-tier devices are in use and also planned for the new DGs? Aim to at least go for 10% cache-capacity ratio but if you have a proportionally large amount of data that is frequently warm and/or IO profiles that are bursty and/or write-intensive then go for bigger (e.g. 600GB for a 3TB DG).

If you don't have pre-deployment HCIBench baselines etc., you can still get a good idea of the current workload and whether it is taxing the current storage by delving into the vSphere Performance charts (Cluster level to start and then host level for more granular information).

Bob