Network Configuration w/ Load Balancing

[khughes]

While playing around in my R/D lab, I was noticing the network usage and how most of the traffic between a teamed set of 2 nics were only using 1 to push all the traffic through while the other remained idle. I decided to take a look at my production network which has a much larger teaming (4 gig nics) and the same was happening where 99% of all traffic was going out 1 nic and maybe a packet or two was going out the second one.

Is there a breaking point where load balancing kicks in? Is teaming the nics together really load balancing? We haven't seen any major network lags so I'm assuming we're just not pushing enough traffic to warrant ESX to load balance the lines. I took some screen shots of the performance and the vswitch config regarding the teaming.

Thanks for any input on the configuration, or explaination

• Kyle

[admin]

This is dependent on the load balancing method used. I suspect all of this traffic is coming from one virtual machine.

http://www.vmware.com/pdf/vsphere4/r40/vsp_40_esx_server_config.pdf

search for load balancing

Rick Blythe

Social Media Specialist

VMware Inc.

[vmroyale]

Kyle,

I'll refer you to Dr. Ken Cline at http://kensvirtualreality.wordpress.com/2009/04/05/the-great-vswitch-debate%E2%80%93part-3/

Good Luck!

[khughes]

Thanks Brian, so basically according to that article it's doing what it's supposed to be doing, and there isn't enough bandwidth being used to warrant it to spread to the other NICs? Also Rick, although there is an outside possibility that the majority of the network is coming from 1 VM, I highly doubt it as we have a Citrix box with 20+ users, email system, and a database on that host.

Maybe now I have the proof to start breaking up the pNICs that I needed

• Kyle

[vmroyale]

I'm betting its more like that the loads are actually spread over the 4 nics, but that the total loads are so insignificant on 3 of them that you are only noticing the higher number on vmnic0. Its possible that there could be a single (as Rick suggested) or multiple "higher traffic" VMs all utilizing vmnic0. vmnic0 receives are peaked at 1.33 MBps, so the load is still light.

This is also the part where I could preach about having baselines for your virtual machines, but I will spare you today!

[naveenvm]

You have selected "route based on originating virtual port id" as the load balancing method to be implied which means it will send the packets in order of, from which nics they came. In short it won't try to scatter the load across the nics and rather follow the external method used.

The best way you can achieve the required is to configure "ether-channelling" or port aggregation at the physical switch for all the ports. At the same time you have to select "route based on ip hash" at vSwitch load balancing method for ESX. This works best !

NUTZ

VCP 3.5

(Preparing for VCP 4)

[jguzmanr]

just remember than any one tcp conversation will always use only 1 pnic, no matter how many pnics you have teamed. If most of your traffic is from one vm to another vm, and you have a 4 pnic team, you will only get 1gb, not 4gb (assumming that your pnics are 1gb).