Odd networking issue. VMs can reach network, but act as if firewalled.

[Evelmike]

I have an ESXi 4.1 host, running on a Dell R910. I have 10 VMs on this host, all running Windows Server 2008 R2 Enterprise.

This host is on a flat network, with no VLans.

I currently have only one Vswitch, with all hosts, and the Management Network, attached to it.

All VMs are able to connect to the LAN, and are able to reach out to all LAN resources. They are also able to hit the WAN, and browse the internet, download files, etc.

The problem starts, when you try to connect to any of the VMs, from a machine that is not also a VM on the same VMhost.

The VMs can ping/RDP to each other without issue. However, if you are attempting to connect to the VMs from, say, my workstation, all ports show as filtered, as if the machines are behind a firewall.

NMAP reports that the hosts are up, and that all 1000 ports are filtered.

MS Network Monitor shows ARP and SMB reaching the VMs. But if you ping, or attempt to RDP or telnet etc, to one of the VMs, the connect simply times out.

To answer the obvious first: Windows Firewall has been disabled (even the Windows Service itself). There is NO software installed on the machines, i.e. no AV, no third-party Firewall, etc. These are completely fresh, patched Server 2008 R2 Enterprise images.

There is an additional Dell server, on the same subnet, attached to the exact same switch, that is running Server 2008 as well; it is completely, 100% reachable. The problem only exists when you are attempting to reach a VM that is behind the Vswitch, from a machine that is NOT behind that same Vswitch.

Any ideas? This one has me seriously stumped.

Edit: I should add that, the host is reachable via vSphere Client, without issue.

[a_p_]

How did you configure the vSwitch policy and - very important - how did you configure the uplink ports on the physical switch? Maybe there's some kind of port security enabled which needs to be disabled for ESXi!?

André

[weinstein5]

Welcome to the Community - Also check if there us a firewall enabled within the OS of the VM - I have had this happen to me when I did check the security on the windows VM I was trying to connect to -

[Evelmike]

Vswitch policy is nothing special:

There are no such settings available on my physical switches.

[Evelmike]

@weinstein5:

Thanks for the welcome.

Per my original post, there is no Firewall enabled on the Guest OS.

[weinstein5]

sorry missed that -

[a_p_]

There are no such settings available on my physical switches.

What type (vendor/model) of switches do you use? Are these managed switches?

André

[Evelmike]

Yes, they are managed switches.

Switches are TRENDnet TEG-448WSs.

[a_p_]

I just took a quick look into the switch's documentation and was not able to find any hints for how to setup the ports to "access mode". Do you have anybody at TrendNet you can ask and/or do you have another switch (even unmanaged) which you could temporarily use to see whether the issue is related to the physical switch/port settings.

André

[Evelmike]

Unfortunately, I am 225 miles away from the server room in question.

There is one unmanaged D-Link switch down there somewhere. I will see if I can get someone to switch out the TRENDnet for the D-Link, and post back.

I should mention, however, that there is another host on the same TRENDnet switch, a physical dell machine, running Server 2008. That machine has no issues whatsoever.

What could cause such an issue specifically with ESXi 4.1?

[a_p_]

The difference between the physical Windows 2008 Server and the ESXi host is, that the physical switch will see only a single MAC address on the port for the Windows Server, but multiple MAC addresses (physical MAC address as well as the VM's MAC addresses) on the ESXi port(s).

André

[Evelmike]

I guess I just don't understand how it is any different then chaining switches together. I'm a Sys Admin, Switch & Router are not my specific areas of expertise.

Thanks for all the help so far, I really do appreciate it. I'll write back once I have someone swap the switches.

[nielse]

Can you give us the configuration on the VM network ?

[Evelmike]

Sure, let me know if I missed anything:

[a_p_]

Is there any reason why you set the VLAN-ID to 4095 on the VM port group? If not remove the VLAN-ID from the port group settings.

André

[Evelmike]

I tried switching it back and forth earlier, with the same result regardless of the setting. I just hadn't switched it back to NONE.

[mooreka]

Hello,

the picture isn't very clear.  Do I see the Management network (with vmotion, FT and Management traffic) disabled?  If so, check management, so it shows enabled.

If I was seeing things, disregard

Can you tell me if vmware tools is installed and the driver configured/installed from vm settings?  Just to be on the safe side, switch it to e1000 or some other generic driver.  If other VM's are running in the vswitch then we wont worry about it.  How about if the device isn't configured or enabled in the guest?

/kelly

[Evelmike]

VMware tools was installed on the Guests. Guest NICs were already set to e1000.

The devices are definitely configured in the Guest, as they are on the network, able to join/leave Domains, able to browse the WAN, and can connect to any LAN object, so long as the connection originates from the VM.

The D-Link dumb switch is apparently dead; it failed to function at all. We're back on the TRENDnet.

I jumped over to an older host machine, this one running ESX 3.5i. I stood up two new VMs, both Server 2008 R2 Enterprise, same as the other guests. ALL of the other VMs on this older host are functioning fine. The new Server 2008 R2 Ent VMs, however, are now exhibiting the same behavior as the VMs on the ESXi 4.1 host.

If it was an issue with the TRENDnet switch, would it not also affect the existing Server 2003 VMs on the older host?

Could this be a Server 2008 R2 Enterprise + VMware issue here?

This just keeps getting better and better!

[nirvy]

To answer the obvious first: Windows Firewall has been disabled (even the Windows Service itself).

Disabling The Windows Firewall Service will continue to block inbound connections due to boot-time policy (you are effectively unloading the run-time firewall policy and forcing the boot-time policy back on) so this won't do you much good unless you also disable the Base Filtering Engine.  Disabling either service is not recommended and probably not supported by Microsoft.  You may want to re-enable it and instead turn off the firewall for whatever network type from Control Panel > System and Security > Windows Firewall > Turn Windows Firewall On or Off.