Esx Server Networking Config

[ash2008]

Good Afternoon,

We are in the process of evaluating Esx Server 3.5. I'm trying to research the best practices for configuring networking. Could you please post an example of how you have your networking setup for esx server. I have three network adapters and a 2950 switch that I'm using to configure different network configs. Are you teaming all network ports at the physical switch then assigning the nics to different virtual switches for high availability? Is it best practice to set force the speed/duplex of the adapter for the nic and the physical switch? I'm able to use another nic as a standby but I'm unable to get the etherchannel working on the physical switch and the virtual switch. Can anyone point me in the right direction for this type of config? My switch has the following Ios version:IOS (tm) C2950 Software (C2950-I6Q4L2-M), Version 12.1(13)EA1, RELEASE SOFTWARE (fc1) Many thanks to all. :smileyblush:

[BenConrad]

Common practices are as follows:

2 NICS: Create 1 vSwitch and then create multiple port groups for each network & service console, vmotion. Setup 802.1q vlan trunking on the switch. You don't need to start with an Etherchannel

3 NICS: Same as above but assign all 3 pNics to vSwitch0.

3 NICS: Create 2 vSwitches and then create multiple port groups, assign 1 pNice to vswitch0 for vmotion; assign 2 pNics to vSwitch1 for each network & service console . Setup 802.1q vlan trunking on the switch. You don't need to start with an Etherchannel

4 NICS: Create 2 vSwitchs and then create multiple port groups, assign pNic (1 & 3) to vswitch0 for SC and vmotion; assign pNic (2 & 4) to vSwitch1 for each network . Setup 802.1q vlan trunking on the switch. You don't need to start with an Etherchannel

You will want to have redundant connections for the service console.

[azn2kew]

If you have total of 3 NICs, I would use this:

1. pNIC1->SC/VMotion

2. pNIC2-3->Virtual Machine Network and pick one NIC just for SC standby since SC is critical to HA feature it will restart your VMs in 15 seconds heart beat.

I would recommend put more NICs on the server if this is a production and heavy use environment. You need at least 6 NICs total to secure with DMZ, Spare and redundancy on SC/VMotion port groups. But as far as design, I would use 1 NIC for Service Console and VMotion and than use 2 NICs for virtual machine network.

If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!!

Regards,

Stefan Nguyen

iGeek Systems LLC.

VMware, Citrix, Microsoft Consultant

[dpomeroy]

Hello,

You might want to take a look at these two docs:

Virtual Networking Concepts

"Etherchannel negotiation, such as PAgP or LACP — must be disabled because they are not supported."

and VMware ESX Server 3: 802.1Q VLAN Solutions

Best Practices would really require more than 3 NICs, but for evaluation purposes it will work fine. How you configure depends on what you want to test, for example, if you have 2 ESX servers and shared storage and want to test VMotion, then you would end up with a different network config than if you are not going to test VMotion.

To make things easy for a basic eval you could just use one NIC for the Service Console port, and then use one NIC for VM traffic. This way you will not have to do any additional configuration on your physical switches.

Don Pomeroy

VMware Communities User Moderator

[ash2008]

Almost all of our physical servers are configured to use EtherChannel and have a two gig connection. I would like to have a two gig connection to the servers using EtherChannel so far I'm unable to get this working with the 2950 switch. Will the V machine see a two gig connection to the switch?

I have configured vmkernel with two nics both are connected to trunk ports. I'm unable to connect to either port. What am I missing on the switch? I can set the port to a trunk I can't specify the 802.11 for the trunk I think thats the default with this Ios ver I'm using. I don't think i can specify the vlans to use (at the switch) over the trunk port maybe this is the issue. Thanks to all.

[Texiwill]

Hello,

When connecting a pSwitch to ESX you do not really want to use Etherchannel. Consider that you have at least 3 networks to deal with:

• Administrative Network (Service Console)

• vMotion Network

• VM Network

• Storage Network (optional, depends if using NFS/iSCSI)

In general for the best performance and security you will want to have each pNIC for each of the major networks. If you must combine pNICs for failover or load balancing reasons you will be looking at VMware NIC Teaming over Etherchannel. While Etherchannel is certainly supported, VMware NIC Teaming provides better results with ESX.

In your situation you need to determine WHERE you want redundancy. For full redundancy, security, and performance you may need 6-8 pNICS not just 3.

The general thought is that you need redundancy for the Service Console (HA complains if this is not the case), but with a VLAN this can be shared with vMoiton. You also need redundancy for the VM Network. THis is the one your VMs live on. You should also have redundancy at the pSwitch as well. ESX/VI3 is all about redundancy. The more you can build in the better you are.

Minimum # of pNICs is 4. 2 for SC/vMotion, 2 for VM Network. WIth 3 you would use 1 for SC/vMotion and 2 for VMs.

If you add iSCSI Storage or NFS based Storage you are looking at adding another pair of pNICs. SAN based storage uses FC-HBA.

You would configure each pair of pNICs via the vSwitch for failover if it is anything but a VM network. If it is a VM Network you could configure the vSwitch for failover and Load balancing.

Once more, Etherchannel is not the best option when connecting to ESX as you are dealing with 3-4 distinct networks.

I would read the documents dpomeroy listed as they will explain how 802.1q (trunking) works. 802.3ad (etherchannel) does work, but you need quite a bit more pNIC for this. The redundancy you want is from the vSwitch to the pSwitch(es), not from the VM to the pSwitch.

In essence, 3 pNIC works, but you will want more for redundancy. ESX has more than one network. Ignore the vmkernel and look at the vSwitch as a simple Layer 2 switch and design your network with that in mind. The vmkernel is not even part of this connection. The vmkernel connects to a vSwitch. The vSwitches connects through the bridged mode pNICs to the pSwitch. You are concerned now about the uplink from the pSwitch TO the vSwitches you have configured.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[azn2kew]

Here is a great presentation talks about performance and networking piece on page 12 talks about EtherChannel and mentioned not worth the effort.

If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!!

Regards,

Stefan Nguyen

iGeek Systems LLC.

VMware, Citrix, Microsoft Consultant