VSAN POC

[pratt26]

Hi,

I am considering VSAN as a solution for hyperconvered for ROBO. I was curious if anyone had success or limitations with 1GB Links for VSAN traffic and use of the Witness appliance at a remote site such as primary data center and or cloud provider. Thanks

[ChrisKuhns · Accepted solution]

I would at least use two if possible, but I guess it would depend on what you're pushing through. I have done it, but I can definitely say I was sweating bullets. One link can work if everything is working just fine, but if you were to have any issues and disks need rebuilding you would be in a world of pain. Before our new switches came in to offer dual 10 GbE ports I had a four node cluster that had only one 10 GbE link setup and a one 1 GbE link setup for management. Something happened and the traffic shifted all to the 1 GbE link. It wasn't pretty. There wasn't even that much traffic on them, just a handful of VMs at the time. Everything essentially became unusable and we suffered for it.

[pratt26]

Thanks Chris

[pratt26]

Thanks Chris, My understanding is that VSAN could not use more than one connection at a time (even if using Etherchannel) did you have multiple ports bonded and did they use the bandwidth?

[zdickinson]

Good afternoon, it's going to be based on how you setup your vSwitch that vSAN uses.  But yes, by default/out of the box it will only use one link, the other is for redundancy.

There is third party software that installs into ESX that will enable aggregate bandwidth.

Something that has been discussed, but I have not tested, is to do two vSAN vSwitches to each host.  One where NIC_1 is active with NIC_2 is standby and one where NIC_2 is active and NIC_1 is standby.  I have done this for vMotion and had it work well.

Thank you, Zach.

[vtonev]

I have a test cluster, where I reach 1600MB/s read troughput on single LACP trunk lag1, consisting of two 10GbE vmnics.

VDS with active LACP link aggregation on both physical adapters. The is not much information about such configuration as VMware recommends the proved active-standby configuration. Also VMware does not guarantee loadbalancing with LACP but it works in my test config.

[zdickinson]

Good morning, that's interesting.  We implemented LACP with VDS and both 10 Gb NICs active and only saw it using one NIC in testing.  Since we didn't see the aggregation benefit we removed the complexity of VDS and went to standard switching.

Could a be a combination of Host, ESX, switch configuration.  Thank you, Zach.