Hello,
VLAN1 is often a special VLAN... Can you move these VMs to a different VLAN ID and see if things work for you?
Best regards,
Edward L. Haletky
VMware Communities User Moderator
====
Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.
Blue Gears and SearchVMware Pro Blogs: http://www.astroarch.com/wiki/index.php/Blog_Roll
Top Virtualization Security Links: http://www.astroarch.com/wiki/index.php/Top_Virtualization_Security_Links
For the portgroup that these VMs belong to, did you specifically define VLAN1 in the configuration? If you did, then the vSwitch is tagging the network traffic in this portgroup, and the following statement needs to be added to your physical cisco switch: vlan dot1q tag native. The cisco switch is not expecting vlan 1 traffic to be tagged, which is why this statement needs to be added. Also, if you remove the vlan setting from this portgroup, the untagged traffic from this portgroup will go to VLAN 1 as all untagged traffic will go to the native VLAN of the trunk port, which is VLAN 1.
One thing I see missing that can cause ESX not to tag some packets is the Native VLAN. The recomendation is to use a Native VLAN that is NOT gonig to pass over the trunk. This will force ESX to tag every packet that doesn't match the Native.
If that is the only VLAN you're having issues with, can you set up a VM on each host on that VLAN and see if you can send pings between them. If that works, this makes me lean more towards a network misconfiguration than an ESX issue.
Are these other physical servers connected to the same switch or switches as the ESX hosts?
The other physical servers on VLAN121 are not on the same switch as the ESX servers.
Just to add, when we removed the trunking, both VM250 & VM251 are able to ping to the rest of the physical servers.
With the port trunk, I've also tried to create a new vSwitch with just VLAN121 as the port group. And that didn't work either.
~~~~~ To Live Is To Die ~~~~~
I have had this before, basically boiled down to VLAN1 is not supported in vSwitches.
Andy, VMware Certified Professional (VCP),
If you found this information useful please award points using the buttons at the top of the page accordingly.
VLAN1 is supported in vSwitches. Read my earlier post. If you assign VLAN1 to a vswitch portgroup, then your physical switch must be configured to expect tagged traffic on all vlans configured on a trunked port.
Since the other physical servers are not connected to the same switch, then to me it seems like a misconfiguration along the network from the switches for your ESX boxes on up your network. I'm not sure exactly how your network is configured, but it could be as simple as a VLAN 121 missing from the trunk ports between two switches.
The Connection between the 2 switches is trunk and allowed all VLAN.
There's no VLAN1 on the vSwitch as well. Funny thing was, when I created a new vSwitch with only VLAN121, it still couldn't route.
I'm running out of ideas to isolate the problem. Right now, I'm unsure if the problem is with the ESX setup or the n/w.
Though I'm more inclined to think this is a n/w issue, I don't have anything to tell the n/w guy to check (a diff grp is handling the n/w).
~~~~~ To Live Is To Die ~~~~~
Well you have already verified that two VMs on separate hosts on the same VLAN can talk to each other, so that should rule out anything with ESX and the physical switch the ESX hosts are connected to. Without knowing your network layout, it's hard to try and pinpoint where a problem may be, but I still think it's a network issue.
Just to share, here's what I did:
1) Remove "VLAN ALL" from vSwitch1.
2) Remove vmnic1 from vSwitch1.
3) Add back vmnic1 to vSwitch1.
4) Issue a "refresh" for the network adapters, you can now see "VLAN121" on the "Observed IP Ranges".
5) Edit settings for VM250, VM251. Select another VLAN click "OK". Then select back "VLAN121" & click "OK".
6) A few seconds later, able to ping them.
Finally, managed to solve the problem.
~~~~~ To Live Is To Die ~~~~~