mbx369
Enthusiast
Enthusiast

Unable to route for a particular VLAN using port trunking

Hi,
I am running 3 ESX servers on v3.5 update 2.
Currently, I have 2 VMs (VM250 & VM251) that are inaccessible via the network. I had configured both the VMs to use vSwitch1, onwhich the port trunking had been done earlier. I do not face such n/w problems with the other VMs except for these 2, which are on VLAN1.
Below are some of the tests that I had done:
* remove & re-create vSwitch1 and the port groups => problem persists * create a different vSwitch and reassign the vmnic to this new one => problem persists * add persistent route on each VMs => problem persists * add persistent route on each VMs, remove the port trunking and configure to only access VLAN1 => able to ping gateway, term services successful * add persistent route on each VMs and revert to the original port trunking => problem persists
The 2 VMs are on Windows 2003 Ent Server 32-Bit. And the problem is not applicable to any particular ESX host or vmnic. The switch used is a Cisco C4948
A sample of the port trunk config:interface GigabitEthernet1/3 description <ESX host> switchport trunk encapsulation dot1q switchport trunk allowed vlan 1,2,3,4,5,6 switchport mode trunk speed 1000 duplex full spanning-tree portfast end
Appreciate anyone's advise. 😄
~~~~~ To Live Is To Die ~~~~~~
Please awards points if this was useful. :) ~~~~~ To Live Is To Die ~~~~~ VCP3/4/5
0 Kudos
19 Replies
Texiwill
Leadership
Leadership

Hello,

VLAN1 is often a special VLAN... Can you move these VMs to a different VLAN ID and see if things work for you?


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

Blue Gears and SearchVMware Pro Blogs: http://www.astroarch.com/wiki/index.php/Blog_Roll

Top Virtualization Security Links: http://www.astroarch.com/wiki/index.php/Top_Virtualization_Security_Links

--
Edward L. Haletky
vExpert XIII: 2009-2021,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
0 Kudos
jguidroz
Hot Shot
Hot Shot

For the portgroup that these VMs belong to, did you specifically define VLAN1 in the configuration? If you did, then the vSwitch is tagging the network traffic in this portgroup, and the following statement needs to be added to your physical cisco switch: vlan dot1q tag native. The cisco switch is not expecting vlan 1 traffic to be tagged, which is why this statement needs to be added. Also, if you remove the vlan setting from this portgroup, the untagged traffic from this portgroup will go to VLAN 1 as all untagged traffic will go to the native VLAN of the trunk port, which is VLAN 1.

0 Kudos
mbx369
Enthusiast
Enthusiast

Hi,
Sorry, I changed the vlan IDs in the ealier post.
The actual trunk config is as follows:
interface GigabitEthernet1/3 switchport trunk encapsulation dot1q switchport trunk allowed vlan 101,104,106,107,110,112,114,117,121,200 switchport mode trunk speed 1000 duplex full spanning-tree portfast end
The "problem" VLAN is 121. There are no issues with routing other VLANs with this trunking config.
~~~~~ To Live Is To Die ~~~~~
Please awards points if this was useful. :) ~~~~~ To Live Is To Die ~~~~~ VCP3/4/5
0 Kudos
Rumple
Virtuoso
Virtuoso

One thing I see missing that can cause ESX not to tag some packets is the Native VLAN. The recomendation is to use a Native VLAN that is NOT gonig to pass over the trunk. This will force ESX to tag every packet that doesn't match the Native.

0 Kudos
jguidroz
Hot Shot
Hot Shot

If that is the only VLAN you're having issues with, can you set up a VM on each host on that VLAN and see if you can send pings between them. If that works, this makes me lean more towards a network misconfiguration than an ESX issue.

mbx369
Enthusiast
Enthusiast

I had placed the 2 VMs (VM250 & VM251) separately among the 3 ESX host servers. Like kind of a "mix & match" order.
Irregardless of the ESX host, VM250 & VM251 are able to ping each other, but not physical servers that are on the same VLAN (VLAN121).
Does this mean that there's nothing wrong with the ESX? If so, how or what can I tell the network guys to work on? ~~~~~ To Live Is To Die ~~~~~
Please awards points if this was useful. :) ~~~~~ To Live Is To Die ~~~~~ VCP3/4/5
0 Kudos
jguidroz
Hot Shot
Hot Shot

Are these other physical servers connected to the same switch or switches as the ESX hosts?

0 Kudos
mbx369
Enthusiast
Enthusiast

The other physical servers on VLAN121 are not on the same switch as the ESX servers.


Just to add, when we removed the trunking, both VM250 & VM251 are able to ping to the rest of the physical servers.


With the port trunk, I've also tried to create a new vSwitch with just VLAN121 as the port group. And that didn't work either.

~~~~~ To Live Is To Die ~~~~~

Please awards points if this was useful. :) ~~~~~ To Live Is To Die ~~~~~ VCP3/4/5
0 Kudos
atbnet
Expert
Expert

I have had this before, basically boiled down to VLAN1 is not supported in vSwitches.

Andy, VMware Certified Professional (VCP),

If you found this information useful please award points using the buttons at the top of the page accordingly.

Andy Barnes
VCP / VCA-DT / MCITP:EA / CCIA
Help, Guides and How Tos... www.VMadmin.co.uk

If you found this information useful please award points using the buttons at the top of the page accordingly.
0 Kudos
mbx369
Enthusiast
Enthusiast

ok, but the problem is with VLAN 121.
what was the solution you used to resolve the problem u faced? ~~~~~ To Live Is To Die ~~~~~
Please awards points if this was useful. :) ~~~~~ To Live Is To Die ~~~~~ VCP3/4/5
0 Kudos
jguidroz
Hot Shot
Hot Shot

VLAN1 is supported in vSwitches. Read my earlier post. If you assign VLAN1 to a vswitch portgroup, then your physical switch must be configured to expect tagged traffic on all vlans configured on a trunked port.

0 Kudos
jguidroz
Hot Shot
Hot Shot

Since the other physical servers are not connected to the same switch, then to me it seems like a misconfiguration along the network from the switches for your ESX boxes on up your network. I'm not sure exactly how your network is configured, but it could be as simple as a VLAN 121 missing from the trunk ports between two switches.

mbx369
Enthusiast
Enthusiast

ok, i'll check this out first.
Hopefully that's the problem. 🙂 ~~~~~ To Live Is To Die ~~~~~
Please awards points if this was useful. :) ~~~~~ To Live Is To Die ~~~~~ VCP3/4/5
0 Kudos
mbx369
Enthusiast
Enthusiast

The Connection between the 2 switches is trunk and allowed all VLAN.

There's no VLAN1 on the vSwitch as well. Funny thing was, when I created a new vSwitch with only VLAN121, it still couldn't route.

I'm running out of ideas to isolate the problem. Right now, I'm unsure if the problem is with the ESX setup or the n/w.

Though I'm more inclined to think this is a n/w issue, I don't have anything to tell the n/w guy to check (a diff grp is handling the n/w).

~~~~~ To Live Is To Die ~~~~~

Please awards points if this was useful. :) ~~~~~ To Live Is To Die ~~~~~ VCP3/4/5
0 Kudos
jguidroz
Hot Shot
Hot Shot

Well you have already verified that two VMs on separate hosts on the same VLAN can talk to each other, so that should rule out anything with ESX and the physical switch the ESX hosts are connected to. Without knowing your network layout, it's hard to try and pinpoint where a problem may be, but I still think it's a network issue.

0 Kudos
mbx369
Enthusiast
Enthusiast

Thx, hopefully VM Support can find something else.
~~~~~ To Live Is To Die ~~~~~
Please awards points if this was useful. :) ~~~~~ To Live Is To Die ~~~~~ VCP3/4/5
0 Kudos
mbx369
Enthusiast
Enthusiast

I just realised something. I don't see the VLAN121 in the "Observed IP Ranges".
Is this something to be concerned? If so, how do we make the VLAN appear in the list? ~~~~~ To Live Is To Die ~~~~~
Please awards points if this was useful. :) ~~~~~ To Live Is To Die ~~~~~ VCP3/4/5
0 Kudos
mbx369
Enthusiast
Enthusiast

Just to share, here's what I did:

1) Remove "VLAN ALL" from vSwitch1.

2) Remove vmnic1 from vSwitch1.

3) Add back vmnic1 to vSwitch1.

4) Issue a "refresh" for the network adapters, you can now see "VLAN121" on the "Observed IP Ranges".

5) Edit settings for VM250, VM251. Select another VLAN click "OK". Then select back "VLAN121" & click "OK".

6) A few seconds later, able to ping them.

Finally, managed to solve the problem.

~~~~~ To Live Is To Die ~~~~~

Please awards points if this was useful. :) ~~~~~ To Live Is To Die ~~~~~ VCP3/4/5
0 Kudos
mbx369
Enthusiast
Enthusiast

Hi,
Sorry for the late update.
I finally found out where the problem was.
On the switch ports, the following must be set:
spanning-tree portfast trunk
Previously, the "trunk" wasn't there. According to the CISCO IOS, we'll the "trunk" in the STP line to enable STP in trunk mode.
Appreciate your time to discuss the problem.
Cheers 🙂 ~~~~~ To Live Is To Die ~~~~~
Please awards points if this was useful. :) ~~~~~ To Live Is To Die ~~~~~ VCP3/4/5
0 Kudos