how to improve apparent networking bottleneck??

[tlyczko]

We've had intermittent complaints of slowness with Kronos and Great Plains on a two-host ESXi 5.5 cluster.

Storage is all 15k SAS drives running on HP StoreVirtual VSAs...should be no lack of IOPs since each HP DL380 G8 server contains 12 x 600 GB 15k SAS drives.

However, today I noticed looking at realtime networking and noticed that networking is pretty close to 900-1000 Mbps...and the hosts are connected to one HP 2920 1 GbE switch...the switch itself goes out to a SonicWall NSA 3600 with 1 GbE ports...

Each host has 6 NICs, 2 for vKernel and Management Network, 2 for VM Network (production VMs), 2 for SAN (the HP StoreVirtual VSA).

The NICs are teamed in the networking configuration as active/inactive on the vSwitch for the SAN...the respective switch ports for the HP VSA SAN NICs are trunked on the switch.

My questions:

1) For the VM Network both NICs are teamed as active together...is this appropriate?? should they be teamed differently??

2) Also I realized the VM Network switch ports are not trunked on the HP switch -- would this improve the networking usage that we see in these graphs??

3) Or should we upgrade to 10 GbE??

The total quantity of VMs is only 27 VMs total thereabouts...

Thank you, Tom

[jrmunday]

Hi Tom,

It looks like you are hitting the physical limitations of 1Gb ethernet (~125MB/sec throughput), so yes the easy option is to upgrade to 10Gb but this obviously has a cost associated with it.

It's not clear from the charts colors which vmnic is receiving so much traffic (looks like vmnic3), so it would help if you explained further how your networking is configured - including whether you are using standard or distributed switches.

For example;

vSwitch0 - vmnic0,vmnic1 (active/active)

- Management (vmk), vmnic0,vmnic1 (active/standby)

- vMotion (vmk), vmnic1,vmnic0 (active/standby)

vSwitch1 - vmnic2,vmnic3 (active,active)

- Guest Networking (vm), vmnic2,vmnic3 (active,active)

vSwitch2 - vmnic4,vminc5 (active,active)

- iSCSI/NAS storage, vmnic2,vmnic3 (active,active)

Depending on your setup and requirements, distributed switches with a load based teaming policy (not available with standard switches) may help you out without the need to upgrade to 10Gb. In this case, you could use trunk ports to increase the number of available uplinks but this still doesn't overcome the limitations of 1Gb ethernet.

Cheers,

Jon

[tlyczko]

Both hosts are set up the same way with standard vSwitches.

The host has all 1 GbE NICs -- I think four 1 GbE NICs on the motherboard plus a 2nd NIC board for the 5th and 6th NICs.

If necessary the 2-NIC board could be replaced with bigger NIC board and more NICs could be teamed etc.

It's a vSphere Essentials Plus license.

Upon planning and configuring the servers and switch etc. last year, did *not* expect so much network traffic etc. etc.

Meanwhile I will try trunking the switch ports involved with vSwitch2 on each host.

All the networking is 1 GbE networking, the VMs connected to the switch, the switch itself, the switch goes to a SonicWall NSA 3600 with GbE ports, this is connected to a 1 GbE that is connected to the Internet...

vmnic configuration on each of the hosts' vSwitches is the same, as near as I can see

vSwitch0 - vmnic0,vmnic1 (active/active)

- Management (vmk0), vmnic0,vmnic1 (active/active)

- VMkernel (vmk1), vmnic0,vmnic1 (active/active)

vSwitch1 - vmnic2,vmnic3 (active,active)

- iSCSI VSA01 storage, vmnic2,vmnic3 (active, inactive)

- iSCSI VSA02 storage, vmnic3,vmnic2 (active, inactive)

I believe HP says not to team the vNICs.

Upgrading to VSA 12.0. may enable changing the iSCSI to round robin, I'm not sure.

vSwitch2 - vmnic4,vminc5 (active,active)

- Guest Networking (vm), vmnic4,vmnic5 (active,active)

Just three VLANs, default, DMZ, outside

Attached diagram probably helps make sense better.

Thank you, Tom