Bladecenter ESX server Vlan trunking problem

[rongill]

Bladecenter ESX server Vlan problem

ESX, Bladecenter, VLAN

Hello,

The bladecenter was initially setup by a company and allows me to only

create VM's from Vlan 2. The issue is that I am out of Ip's in that

vlan and I wanted the ability to use another address space for the

VM's. I've already created a new vlan 20.

Here is the current config of the switch. All I've added to these configurations from before was sw trunk allowed vlan add 20

Main Switch CISCO 4500

interface Port-channel1

switchport

switchport trunk encapsulation dot1q

switchport trunk native vlan 3

switchport trunk allowed vlan 1-3,20

!

!

interface GigabitEthernet5/39

description BLADECENTER-sw2-p17 Trunk Port

switchport trunk encapsulation dot1q

switchport trunk native vlan 3

switchport trunk allowed vlan 1-3,20

channel-group 1 mode on

!

interface GigabitEthernet6/2

description BLADECENTER-sw2-p17 Truck Port

switchport trunk encapsulation dot1q

switchport trunk native vlan 3

switchport trunk allowed vlan 1-3,20

channel-group 1 mode on

!

Switch config - Bladecenter sw1

vlan 2

server

vlan 3

WAN

vlan 20

server2

interface Port-channel1

switchport trunk native vlan 3

switchport trunk allowed vlan 1-3,20

interface GigabitEthernet0/17

switchport trunk native vlan 3

switchport trunk allowed vlan 1-3,20

macro description cisco-switch

channel-group 1 mode on

!

interface GigabitEthernet0/18

switchport trunk native vlan 3

switchport trunk allowed vlan 1-3,20

macro description cisco-switch

channel-group 1 mode on

blade switchport where host is contained

interface GigabitEthernet0/9

description blade9

switchport trunk native vlan 2

switchport trunk allowed vlan 2,20

switchport mode trunk.

The ESX host IP is in Vlan 2

End result is I can't use any vlans from that new range. There is no Vlan ID or tagging setup.

Thanks in advance.

[Rockapot]

Do you have a PortGroup setup for the new VLAN yet?, Your switch config looks like it is presenting the VLAN to the Bladecentre.

If you have not already then you should create a PortGroup on your VM vSwitch and then tag it to VLAN20.

Carl

[Rockapot]

Can you also provide a screenshot of your ESX network configuration which is available via the VI client on the ESX hosts configuration tab?

[rongill]

I did create the portgroup last night .. but still no go. Here is a ss of the config tab

[Rockapot]

Which interface is connected to vmnic0 out of the one's you mentioned below?

interface GigabitEthernet5/39

interface GigabitEthernet6/2

interface GigabitEthernet0/17

interface GigabitEthernet0/18

interface GigabitEthernet0/9

I see that GigabitEthernet0/9 has no 802.1Q vlan trunking in place..

You will need 802.1Q vlan trunking in place on vmnic0 to be able to enable ESX to tag to the VLAN on the physical switch.

A good artcile to read is here: , you should review the differences in EST, VGT and the more commonly used mode of VST..

On a side note, and with out going to much in to detail on the rest of your environment you should ideally have your ESX hosts in a VLAN seperate to that of your VM's..

Carl

[rongill]

It is on GigabitEthernet0/9. It is setup by default trunk method

Name: Gi0/9

Switchport: Enabled

Administrative Mode: trunk

Operational Mode: trunk

Administrative Trunking Encapsulation: dot1q

Operational Trunking Encapsulation: dot1q

Negotiation of Trunking: On

Access Mode VLAN: 1 (default)

Trunking Native Mode VLAN: 2 (Server)

Voice VLAN: none

Administrative private-vlan host-association: none

Administrative private-vlan mapping: none

Administrative private-vlan trunk native VLAN: none

Administrative private-vlan trunk encapsulation: dot1q

Administrative private-vlan trunk normal VLANs: none

Administrative private-vlan trunk private VLANs: none

Operational private-vlan: none

Trunking VLANs Enabled: ALL

Pruning VLANs Enabled: 2-1001

Capture Mode Disabled

Capture VLANs Allowed: ALL

Protected: false

Unknown unicast blocked: disabled

[Rockapot]

There's the issue then.., yeah you will need dot1q enabled on the port to enable the vSwitch to tag to it.., I'm not a cisco guy so its probably best you have a word with one of the networking team if you have one.

Once the port is set to dot1q then your VM that is assigned to the PortGroup (tagged to VLAN20) should be able to communicate with other servers on that VLAN etc..

Additional info on seting dot1q on the port can be found here: http://www.internetworkpro.org/wiki/IEEE_802.1q

I usually just direct the network team to make the changes to the network for me hence I dont usually work on the cisco devices myself..

Hope this helps

Carl

[Rockapot]

Just noticed you updated one of your forum threads with more info on interface 0/9. Trunking appears enabled according to the output however your original posting shows it is not.

I am sure another forum member with some more Cisco knowledge will help out shortly.., Can anyone help rongill on ensuring dot1q is enabled on Interface 0/9?

Carl

[rongill]

Bladesw2#sho int trunk

Port Mode Encapsulation Status Native vlan

Gi0/9 on 802.1q trunking 2

Gi0/15 on 802.1q trunking 1

Po2 desirable 802.1q trunking 3

Port Vlans allowed on trunk

Gi0/9 2,20

Gi0/15 1,3

Po2 1-3,20

Port Vlans allowed and active in management domain

Gi0/9 2,20

Gi0/15 1,3

Po2 1-3,20

Port Vlans in spanning tree forwarding state and not pruned

Gi0/9 2,20

Gi0/15 1,3

Po2 1-3,20

[rongill]

I checked with my network team and I am dot1q enabled.

[Rockapot]

As dot1q is enabled, the VLAN is presented and the PortGroup tagged then it might be worth setting up a ping from the VM machine on VLAN20 to a physical server on the same VLAN outside of the VM environment and then get the Network guys to see where the communications are getting stopped. They should be able to see where the traffic is getting blocked

Carl

[JeffDrury]

How are you determining that the port group for VLAN20 is not working? From the screenshot of your ESX vSwitch config there are no VM's assigned to the port group. When you add a VM NIC to the port group does it see the other devices on the VLAN? Also it looks like cisco switch is configured correctly, but you mention that this is on a bladecenter device. Are there dedicated NIC's for each ESX host or is there a central interface that all blades share? If this is the case you may need to configure the blade infrastructure to pass the VLAN tags to the blades. It may be the blade infrastructure that is blocking VLAN tags.

[rongill]

I haven't determined that. The screenshot doesn't even show the current VM's on Vlan 2 anymore as the other hosts do.

When you add a VM NIC to the port group does it see the other devices on the VLAN?

What do you mean by other devices?

There is a dedicated NIC for each ESX host.

[JeffDrury]

Yes you need to add the VM NIC to the port group or it will not be able to communicate on that VLAN. To test this you could create a test VM and assign the NIC to the port group for VLAN 20 or add another virtual NIC to an existing VM and assign it to port 20. It is probably safer to create a test VM and assign the vNIC.

When I mention 'other devices' I am referring to the networking component of the bladecenter servers. I am not familiar with IBM's particular technology but some blade systems have a central networking device that shares and manages the NIC's. In theory this means that two blades could be using the same NIC on the blade chassis but to the blades it looks like they each have a dedicated NIC. There is usually a configuration console for the blade chassis that lets you configure these settings. Anyone have IBM Blade Center experience that could back this up?

Really I think this is likely an issue with not assigning the vNIC to the port group. Try that first and if it doesn't work you may want to look deeper at the blade infrastructure.

[rongill]

Hi Jeff,

You maybe right. I didn't know you had to assign the VM to a particular portgroup in the host settings. Just trying it out. Will keep all updated. Thanks !

[Rockapot]

I must admit I had thought you had assigned the VM to the PortGroup when you had setup the PortGroup earlier

Carl

[rongill]

that did it guys .... thanks again!

When i presented that screenshot with No VM's showing on either vlan that should have been the trigger.

[Rockapot · Accepted solution]

Great..

Remember to award points for helpfull or correct answers

[JeffDrury]

Good to hear it's working.