IOPS consideration

[suhag79]

Hi,

We are going to setup remote site with vSAN 6.5.  Expected IOPS requirement from app team is 15000 IOPS for all the VMs to be hosted, majority of 15K IOPS are from one VM only.

I read that vSAN ready node HY-6 series (CISCO C240 M4SX) can provide up to 20K IOPS. SO my question is, how to consider this 20K IOPS, is this for overall combining all 4 vSAN node ?

For example, if i have only one VM  running on vSAN  node-1, can that VM get whole ~ 20K IOPS ?

Thanks,

[TheBobkin]

Hello suhag79​,

"Expected IOPS requirement from app team is 15000 IOPS for all the VMs to be hosted"

Weird - it showed as 9k IOPS a moment ago, what changed? :smileygrin:

"I read that vSAN ready node HY-6 series (CISCO C240 M4SX) can provide up to 20K IOPS"

Source of where you read this please.

This would almost certainly be IOPS per node as there is no way of knowing how many hosts in a cluster - e.g. if you had 8 nodes in a cluster instead of 4, a per-cluster performance estimate would make no sense.

Do note that these 'up to' figures are usually done using the optimal configuration (e.g. more expensive/better disks, controllers etc.) unless otherwise specified (e.g. for a Ready-Node model that has no configurable options such as better disks).

This also assumes optimum workload characteristics such as IO size and usage-pattern.

Typically don't take any 'up to' performance as gospel - test the actual IOPS this cluster is capable of using a tool such as HCIBench.

"For example, if i have only one VM  running on vSAN  node-1, can that VM get whole ~ 20K IOPS ?"

If you have a non-striped FTT=1 Object this could at most use the IO performance of 2 nodes (as only 2 would be storing data-components), *potentially* if this was striped in such a manner across all hosts (e.g. 2x RAID0 mirrors each over 2 hosts, no witness components) this could use 4 hosts output but I don't think there is a way to force stripe-placement as such.

There are mechanisms in place to not starve other VMs of resources but if there is not a huge amount of contention, yes one VM can use a larger amount of storage resources than others.

Bob

[suhag79]

Hi TheBobkin​

We have checked with app team, if the IOPS would be same for development VMs and they says, it would be near to same and thats why figure got changed

Here is the source, where it is mention that IOPS can be up to 20 K.

http://partnerweb.vmware.com/programs/vsan/Virtual_SAN_Hardware_Quick_Start_Guide.pdf

i completely agreed on below point.

==========================================================

Do note that these 'up to' figures are usually done using the optimal configuration (e.g. more expensive/better disks, controllers etc.) unless otherwise specified (e.g. for a Ready-Node model that has no configurable options such as better disks).

This also assumes optimum workload characteristics such as IO size and usage-pattern.

Typically don't take any 'up to' performance as gospel - test the actual IOPS this cluster is capable of using a tool such as HCIBench.

====================================================

How much write IOPS would be impacted if we use 1 GB network. I know read IOPS will not rely  on back end network but i am more  worried about write IOPS if we go for 1 GB with FTT=1 ???

FYI - our 70% load would be read IOPS.

[TheBobkin · Accepted solution]

Hello suhag79​,

Well that Ready-Node is specific about what disk-class are used in it so this 'up to' only relies on whether the IO and workload are perfectly suited for what vSAN is optimized for - 70/30 Read/Write is pretty typical so shouldn't see any issue with that aspect.

"How much write IOPS would be impacted if we use 1 GB network. I know read IOPS will not rely  on back end network but i am more  worried about write IOPS if we go for 1 GB with FTT=1 ???"

It generally matters more about the switch really - Hybrid runs fine on dedicated 1Gb uplinks so this shouldn't be a bottleneck, however there can be a HUGE difference in capabilities of a cheap 1Gb switch vs an expensive one.

Bob

[suhag79]

Thanks TheBobkin​,

Since our read IOPS work load is more than 70-80% , i think, we can go with 1 GB network.

Thanks again for your help.

[Linjo]

Read IOPS will also be depending on the network speed of course. If the VM is running on one of the 2 hosts that does not have the the VMDK object then it will get all that data over the network unless its in the local read cache. The more hosts you have in the cluster the higher probability that the VM is not running on one of those 2 hosts.

10GbE switches are cheap now, you can get a decent one for 3000USD. Its a no-brainer, get yourself some decent switches.

// Linjo

[elerium]

With the exception of testlabs and the lightest of workloads (maybe a remote office), i really don't think 1gb networks are suitable for VSAN anymore. Remember with FTT=1 that all your data is being duplicated across hosts as well, factor in read VM activity and you'll find there's barely any bandwidth left for disks. I also imagine during resync events (you won't see this activity on a new cluster, but as the disks fill up you will see some resync eventually) on a 1gb network you're going to see a very large drop in disk & VM performance and this can be for sustained long periods of time.

[suhag79]

Thanks,

Just one observation.

i seen 7000 IOPS on vSAN which run on 1GB network. Out of 7000, 6000 IOPS are for read operation. I have checked the network traffic on vSAN kernel interface at the same time and it was around 60 Mbps.