VMware Cloud Community
Dalmata71
Contributor
Contributor

teaming with four nics

Hi ,

I have a virtual infrastructure with Esx 3.5 (Blade server).

Each Blade have 4 nic ( in the back plane)and I want to configure networking in this environment ( vswitch ) for a best Load balancing and failover.

What's the better solution ?

Single vSwitch with four nic in teaming configuring etherchanel ? 2 virtual switch o else ?

Thank you

0 Kudos
27 Replies
ChrisDearden
Expert
Expert

are you wanting to use the same 4 pnics for all your traffic ? (Vmotion , Service Console , Virtual machine Network and (potentially ) storage ? )

If this post has been useful , please consider awarding points. @chrisdearden http://jfvi.co.uk http://vsoup.net
0 Kudos
demz
Expert
Expert

It depends how much VMs you will run on these blades and how I/O intensive they will be.

But two vmnic can be enough for VM network.

So you can do something like that :

vmnic0 : COS (VMkernel redundancy)

vmnic1 : VM

vmnic2 : VM

vmnic3 : VMkernel (COS redundancy)

All that on one vSwitch but different portgroups (and VLANs)

You can use load balancing instead of etherchannel, route based on port ID (default) is a good choice.

Hope this helps,

0 Kudos
Dalmata71
Contributor
Contributor

Yes....

I know is bad idea but my customer would like to take advantage of 4 pnic for increase of bandwith and failover.

Can I have a teaming of 4 pnic with etherchanel configuration on phisical switch, setting ip hash on virtual switch, and trunk (802.1q) toghether?

What configuration can you suggest me ?

0 Kudos
Dalmata71
Contributor
Contributor

Thanx for reply,

in your configuration I' will have a traffic across same pnic true?

Instead ,when I configure etherchanel with ip hash , for comunication with multiple client the traffic is over more pnics.

0 Kudos
Texiwill
Leadership
Leadership

Hello,

Check out http://www.networkworld.com/community/node/35854 for assistance with 4 pNICs.


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

SearchVMware Blog: http://itknowledgeexchange.techtarget.com/virtualization-pro/

Blue Gears Blogs - http://www.itworld.com/ and http://www.networkworld.com/community/haletky

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

--
Edward L. Haletky
vExpert XIV: 2009-2023,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
0 Kudos
demz
Expert
Expert

The VM's traffic will be loadbalanced between the two ports (vmnic1 & vmnic2)

0 Kudos
khughes
Virtuoso
Virtuoso

Are you really going to utilize the bandwidth of 4 gig NICs? We're trying to move away from 4 teamed NICs that run everything and we probably wouldn't stress the load of 1, let alone 4. I would look at the amount of bandwidth you would be consuming before you assign so many gig nics around.

Utilizing 802.1q would be an advantage if you wanted to go the route of using all 4 teamed.

  • Kyle

-- Kyle "RParker wrote: I guess I was wrong, everything CAN be virtualized "
0 Kudos
atbnet
Expert
Expert

I regard the following setup as the best for performance and reliability using 4pNICs and 2 physical switches....

Each ESX server has 4 Physcial Nics (pNICs).

Each server has the pNics assigned to vSwitches as follows:

vSwitch0 vmnic0 + vmnic2 (used for VM traffic) with IPhash load balancing - all adaptors active

vSwitch1 vmnic1 + vmnic 3 (used for vMotion and SC traffic) with IPhash load balancing - all adaptors active

A pNIC from each vSwitch goes to a different physical switch for failover.

All links are 802.1q vlan trunk ports.

Andy, VMware Certified Professional (VCP),

If you found this information useful please award points using the buttons at the top of the page accordingly.

Andy Barnes
VCP / VCA-DT / MCITP:EA / CCIA
Help, Guides and How Tos... www.VMadmin.co.uk

If you found this information useful please award points using the buttons at the top of the page accordingly.
Texiwill
Leadership
Leadership

Hello,

the best security is using multiple physical connections for the various networks involved. Within only 4 pNICs in the chassis you have to make some very hard decisions dealing with security. 802.1q does not play into the discussion on security, but it does play into the discussion when you include performance and redundancy into the security discussion. Use of 802.1q ends up being a trust issue within your physical network and hence your virtual network.


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

SearchVMware Blog: http://itknowledgeexchange.techtarget.com/virtualization-pro/

Blue Gears Blogs - http://www.itworld.com/ and http://www.networkworld.com/community/haletky

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

--
Edward L. Haletky
vExpert XIV: 2009-2023,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
0 Kudos
kurtwest
Contributor
Contributor

My environment is using blades and my ESX hosts have 4 pNICs. Here is what we have and it works well:

vSwitch0 (COS and VMKernel): vmnic0 & vmnic1 each connected to a different physical switch are active on the vSwitch

Service Console and VMkernel portgroup have "Override vSwitch failover order" checked.

Service Console port group has vmnic0 as primary and vmnic3 as standby

VMkernel port group has vmnic3 as primary and vmnic 0 as standby

vSwitch1 (VM Network) - vmnic1, vmnic2 each connected to a different physical switch are active on the vSwitch

We have several VLANs trunked to this vSwitch and each VLAN is a portgroup and we DO NOT override the switch failover settings.

0 Kudos
Dalmata71
Contributor
Contributor

Hi,

I like your solution, but I've any questions :

1) When you set IPhash load balancing it is mandatory to configure on physical switch etherchanel (802.3ad)? If yes , i must aggregate port in the first physical switch where nic0 and nic2 are linked and in the second where nic1 e nic3 are linked?

2) Can I have port of physical switch with etherchanel and trunk together ?

Maybe my questions are stupid but i want understand because I'm very confused.

Thanx a lot

0 Kudos
Texiwill
Leadership
Leadership

Hello,

There is no need to configure etherchannel, while the vSwitch does support 802.3ad, NIC Teaming within the vSwitch is not etherchannel.


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

SearchVMware Blog: http://itknowledgeexchange.techtarget.com/virtualization-pro/

Blue Gears Blogs - http://www.itworld.com/ and http://www.networkworld.com/community/haletky

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

--
Edward L. Haletky
vExpert XIV: 2009-2023,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
0 Kudos
Dalmata71
Contributor
Contributor

Hello,

excuse me I've your book and at page 264 you wrote:

IP load balancing requires the setting up of EtherChannel or 802.3ad on the

pSwitch for the pNICs in the NIC team.

I understand that at virtual switch level i must set in load balancing setting "Routed based on IPHash" , instead, on physical switch ,is mandatory to set EtherChannel or 802.3ad.

It's true?

Thank you

0 Kudos
depping
Leadership
Leadership

I don't see a reason why not to use etherchannels and ip-hash. have done this various times when a customer specifically requested it. i agree that it will probably not give you additional bandwidth in any way.

Duncan

Blogging:

If you find this information useful, please award points for "correct" or "helpful".

jo_strasser
Enthusiast
Enthusiast

Hi!

I have nearly the same coniguration of you in my datacenter...

Whitch Blade Systems of which Vendor have U installed (HP, Dell, IBM...)??

If U have your setup with HP c-Class Blade Enclosures, there it will give some "VMware ESX on c-Class Blades" WhitePapers for the best Configuration...

This WhitePapers will be used by HP for internal use... but I can upload this. It shouldn´t be the problem...

There are differences between Blade Switch Modules or Blade VirtualConnect Modules (Bridges)...

In my environment, I setup my Blade Centers with VirtualConnect Network and FibreChannel Modules... (4x10GB Uplinks to LAN, 8x4GB to SAN).

Please let me know your vendor and configuration... and which switches are U have behind your Blade Enclosures (stacked?)!

Thanks!

best regards,

Strasser Johannes, VCP, CPSA



Johannes Strasser / SDDC Architect @ Porsche Informatik GmbH
Twitter: @jo_strasser
0 Kudos
Texiwill
Leadership
Leadership

Hello,

excuse me I've your book and at page 264 you wrote:

IP load balancing requires the setting up of EtherChannel or 802.3ad on the

pSwitch for the pNICs in the NIC team.

Not my book. P 264 covers NTP, not using etherchannel.

I understand that at virtual switch level i must set in load balancing setting "Routed based on IPHash" , instead, on physical switch ,is mandatory to set EtherChannel or 802.3ad.

However if you wish to use 802.3ad this is correct. My statement is that you do not need to use 802.3ad for the benefits of load balancing to take place. 802.3ad could load balance incoming, while NIC Teaming can load balance outgoing.


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

SearchVMware Blog: http://itknowledgeexchange.techtarget.com/virtualization-pro/

Blue Gears Blogs - http://www.itworld.com/ and http://www.networkworld.com/community/haletky

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

--
Edward L. Haletky
vExpert XIV: 2009-2023,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
0 Kudos
Dalmata71
Contributor
Contributor

Hi,

my vendor is DELL and model is Dell Power Edge M100e , instead the I/O module :

Best regards

0 Kudos
Dalmata71
Contributor
Contributor

Hi,

sorry, for me it's very difficult to understand because I have read a lot of article and my impression is that i need 802.3ad when using IP hash because the pswitch needs to understand that packets from the same MAC address (the VM) could appear on multiple ports.

Thanks

Best regards

0 Kudos
Dalmata71
Contributor
Contributor

Hi,

my impression is that 802.3ad is necessary when using IP hash because the pswitch needs to understand that packets from the same MAC address (the VM) could appear on multiple ports.

I've read a lot of article and I understand this, it's correct?

Thanks

best regards

0 Kudos