Re: EtherChannell NIC's on Managment Network

mackinsBNP · ‎08-08-2011

Hi

I have setup my ESX networking like below. I have 3 NIC's assigned to a vSwitch called vSwitch0. I would like to use just 1 vSwitch in my setup and have both "Management Traffic" and "Virtual Machine Traffic" on this vSwitch. I am not using vMotion. I have setup 2 port groups on my vSwitch, 1 for each of my VLANS that I will be using in my virtual environment. The vSwitch is Set to loadbalance and uses "Route Based on IP hash". All 3 of my NIC ports are trunked. Once my network admin does the configuration on the physical switch and bundles my 3 NICs into an etherchannell(3GB). I loose connectivity to the host and do not regain connectivity until he un-bundles them. He is using the same load balancing protocol on his side "Route Based on IP hash" and has everything setup as it should be. Does anyone have any idea why this happens.

> Can VMKernel port and Virtual Machine traffic exists on one vSwitch. I assume the answer to this a definite yes....!!!

> Can a VMkernel port exist as part of an ethechannell....???

I have done some testing and have createded a seperate vSwitch. I have removed the vmkernel NIC from the etherchannell and moved it to this new switch. Everything works fine. However once I move it back to a vSwitch and make it park of the the ehterchannell I lose connectivity.

I have read through numerous posts and whitepapers and all say my vSwitch is set correctly and I have also confirmed the configuration used for the etherchannell on the physical switch is correct.

Can ANYONE offer any help/solution to this. I must have only 1 vSwitch in my setup and do not want to seperate Management and Virtual machine traffic.

.

SteveFuller2011 · ‎08-08-2011

Hi,

Does your network admin see any errors or SYSLOG message on the switch when he bundles the ports into an EtherChannel? Note that the port-channel interface on the switch must be configured as a static EtherChannel as LACP is not supported on ESX unless you're using the Nexus 1000V.

If these are Cisco switches you'll need something along the lines of the following:

interface <interface_number>

switchport trunk encapsulation dot1q

switchport trunk allowed vlan <vlan_range>

switchport mode trunk

spanning-tree portfast trunk

spanning-tree bpduguard enable

channel-group <etherchannel_number> mode on ! This must be set to on, rather than active or passive

!

The switch ports should also not have the command channel-protocol lacp.

Regards

mackinsBNP · ‎08-09-2011

Thanks for the reply Steve, much appreciated. The config on my switch look similar to yours although there are some differnces. I will check these with my network admin and see what he says(he's gone home for the day). The main difference is the two lines "spanning-tree portfast trunk" and "spanning-tree bpduguard enable", not sure if this makes much of a difference to the overall result. Also the "mtu 9216" is not present on your config.

Will post back as soon as I talk to my admin.

switchport
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 100,110,121
switchport mode trunk
mtu 9216
no ip address
speed 1000
duplex full
channel-group 12 mode on

If anyone else has any suggestions of flashes of genius please post them.

SteveFuller2011 · ‎08-09-2011

Hi,

So when you say you loose connectivity, is that to the management IP of the ESX host? Looking at the Cisco switch configuration you're only allowing VLAN 100, 110 and 121. That's fine... no point in allowing more VLANs than is required and running on the host.

When I look at your vSwitch configuration I can see VLAN 110 and 121 assigned as portgroups, but I don't see VLAN 100 anywhere. If this is the VLAN you've assigned for the VMkernel port i.e., the ESX management, then you'll need to add VLAN 100 to the VMkernel configuration in the vSwitch.

This is the setup on my host.

And the configuration on the Cisco switch is switchport trunk allowed vlan 2,10.

Regards

mackinsBNP · ‎08-09-2011

Hi Steve

The VMkernel is on the 121 VLAN, so its covered in the 121 port group. I just didnt create the VLAN-100 port group as I had been removing and re-creating vSwitches all day so just left it out. Being lazy pretty much.

Your setup looks like exactly what I am aiming to acheive. Can I be a pain and ask you to send me screenshot of the below items. If you dont have time its fine but would really like to compare the setup on my end to the setup you have. I suspect this is some sort of network (physical switch) issue as I have tried almost every configuration possible on the vSwitch side.

> Config properties of vSwitch0

> Config properties for your WinXP Client Port group,Management Network port group,VM network port group

> Full config of your physical switch to accomodate this setup.

Sorry for being such a pain, but I would really appreciate if you could send these.

Thanks a lot Steve.

mackinsBNP · ‎08-09-2011

I know I am really pushing it now but can you also send me screenshots of the NIC teaming config you have for each of your port groups.

Thank you so much.

SteveFuller2011 · ‎08-09-2011

Hi,

I've included the screenshots and switch configuration as requested, but if your VMkernel port is on VLAN 121, then aren't you missing the VLAN ID from port propoerties of the Management Network?

If you look at the screenshot for the management network below, you'll see I've got VLAN 2 assigned, which is the VLAN for the 192.168.1.0/25 subnet which my VMkernel IP address is from.

Anyway, I think this was what you were after.

vSwitch and portgroup properties:

Teaming properties:

Regards

mackinsBNP · ‎08-10-2011

Hey Steve

Thanks so much for the info, it really is appreciated, I didnt get a chance to do a comparrison of the setups yet but have noticed that the below differs from yours. Will do some testing today and let you know how I go.

Thanks again for the info you will be blessed with good looks and riches for your kindness ; -)

mackinsBNP · ‎08-25-2011

Steve

Are you using Cisco 6509's to acheive the above setup. If so can you tell me your IOS version. This is the only thing that we have no checked. Everything else we have setup exactly like yours and we are still having the same issue. IOS is our last hope.

SteveFuller2011 · ‎08-26-2011

Hi,

In my lab I've a Cisco Catalyst 4948 with 12.2(53)SG1. I've loaded a file with show version and a few other commands from the Cisco switch that you may be able to use to compare to oyur setup.

Good luck.

Regards

mackinsBNP · ‎09-05-2011

Hey Steve

How are you. Do you have your native VLAN ID defined on any of your vSwitches or Port groups in your setup...??

Cheers

SteveFuller2011 · ‎09-05-2011

Hi,

I'm good thanks. Sorry to see you've still not got your connectivity up and working.

The native VLAN on the EtherChannel trunk is the Cisco default i.e., VLAN 1, and this is not configured on any of the port groups within the vSwitch.

The Management Network is part of VLAN 2, which is configured on the VMkernel port; this can be seen in the screen shot of my post from Aug 9, 2011 12:54 AM. Note that VLAN 2 is entered as the VLAN id as part of the configuration of the management network on the console (F2 -> Configure Management Network -> VLAN -> Enter the VLAN id). This is shown in the Set the VLAN id on Page 29 of the ESXi Setup Guide (http://www.vmware.com/pdf/vsphere4/r40/vsp_40_esxi_i_vc_setup_guide.pdf).

Hope that helps.

Regards