Problem = Bladecenter Trunk setup CIGESM - setting the VLAN ID (optional) field, breaks communication

[zthiel]

I have an ESX server installed on a bladeserver. The bladecenter chassis has (4) eth modules.

For example

Bladecenter eth0 - vm traffic

Bladecenter eth1 - vm traffic

Bladecenter eth2 - service console

Bladecenter eth3 - vmotion

Within ESX I have (3) vSwitches setup, one vSwitch has (2) nics assigned (eth0 and eth1) carrying the vm traffic. Another vSwitch is setup with eth2 carrying the service console, and the last vSwitch is setup with eth3 carrying Vmotion.

Lets take the service console vSwitch as an example, as soon as I assign the VLAN ID to anything other than "None" we loose all abilities to communicate with the IP address assigned to the service console of that ESX server. If I put it back to "None" then communication is restored. I have also tried this with the Vmotion vSwitch and experiance the same results.

Ideas?

The config inside the CIGESM is using trunking and has the proper vlan assigned.

[kjb007]

You have to check the port config. If the port config includes access port and trunk mode, then removing the vlan tag will remove the server from the network. Also, make sure that the trunk is not including a native vlan which is the same as the vlan you are trying to have the service console on. In other words, change the native vlan to something other than the vlan you want to be on, or change the vlan to something other than the native vlan. The 2nd is better as the native VLAN is sometimes not changeable on the embedded switches, and the management of the blade sometimes goes through there as well.

-KjB

[lamw]

Now, when you say you have Trunking? Are you speaking in terms of having multiple VLANs assigned to a single switchport? This can have 2 definitions depending if your coming from a Cisco or Dell world or even other switch vendors. Cisco Trunking is defined as having multiple VLANs span a single switchport that have VLAN IDs, in this case if you're attaching your vSwitch to that single Trunk connection, you'll need to create portgroups (basically labels that you can define VLAN IDs), so basically if you leave it to none, it'll use the native VLAN but it sounds like in your case the switchport is not trunk because if you change the VLAN ID it breaks connection, either that or you're providing the incorrect VLAN ID for your Service Console network. The other known Trunking, known in the Cisco world which is Etherchannel, link aggregation, this will have a single network/VLAN but takes multiple NICs and aggregates the traffic to have multiple streams but each stream will not execeed the speed of the NIC, aka if you connect 3 1gbps NICs, you will not get a throughput of 3gbps, but you can have more streams at 1gbps max.

[zthiel]

Sorry I may have stated my example incorrectly.

On my core switch I am connecting (4) interfaces to my bladecenter eth module which also has (4) ports. Then am running LACP to combine them into one pipe. So I have the LACP setup on the core and a port channel defined on the bladecenter switch and all (4) interfaces are in the port channel. The vlans over this LACP connection are 333 and 334 for example. Then on the bladecenter switch g0/4 has the following config:

interface GigabitEthernet0/4

description blade4

switchport trunk native vlan 333

switchport trunk allowed vlan 333

switchport mode trunk

spanning-tree portfast trunk

spanning-tree bpdufilter enable

My port-channel config is:

interface Port-channel1

description EtherChannel-To-6500

switchport trunk native vlan333

switchport trunk allowed vlan 333,334

switchport mode trunk

flowcontrol send off

And my service console IP address is 10.1.333.1.

As soon as I enter in 333 for my VLAN ID in the Service Console vSwitch communication breaks, leaving it at none does not halt communication. Hope this helps.

[kjb007]

That is your problem. By default, cisco will not tag native vlan. Change your port config so that native vlan is not 333. Use 1. Then, add 333 to your service console, and you should be golden.

Another question? How did you get "333" as an allowed number in your octet? The 8 bits allow only for 255.

-KjB

[zthiel]

Sorry I copied and pasted from something else. It should be 210 and 211. I changed the switchport allowed native vlan 1 on the g0/4 interface and it did not work. See the example.jpg attached, the drops were when it was at "1" and once I changed it back to 210 communication resumed.

[lamw]

I'm sure it's all coincidence, but my personal ESX Server at home actually has an internal network that has VLAN tag of "333" which is pretty hilarious. The ID's are allowed to go from 1-2049 or something crazy like that. The limit is not at 255, we have other's that span in the thouands.

[kjb007]

I didn't mean the ID, but the third IP address octet that was posted.

-KjB

[kjb007]

Ok, now I am confused. If you add 210 to your vlan # on your service console, and then change the native vlan, does it all work as expected? In order for 802.1q tagged vlans to work, either both sides have to tag the vlan, or neither side does. If one side is tagging, and the other is not, it will not work. One side will send traffic and expect a tagged vlan for return traffic. The other side will not tag and will accept the traffic, but will not tag back the return traffic, so the communication breaks.

Cisco by default will not tag native vlan. In your case, if you're not adding the tag, and cisco is not expecting a tag, all traffic works fine. When you add a tag, and the other side is not expecting it, the traffic breaks down, which is what you are experiencing.

Hope that makes sense.

-KjB