Hosts connect to vCenter, then disconnect after approx 2 minutes

[hbtadmin]

I have a vCenter 4.1 environment with 6 hosts all running ESXi 4.1 and all is good.  I purchased additional licenses (existing 6 are Enterprise Plus and I bought 2 Enterprise licenses) and I am trying to add 2 additional hosts.  One of the new servers is an existing server running ESXi 4.0 and the other new server is a brand new server and I installed 4.1 U1 on it.  Both servers are HP DL360s, existing is a G6 and the new one is a G7.

I first tried to add the new server.  I connects fine, installs the agent, and looks fine for about 2 minutes and then goes to not responding.  I can click on Connect and it connects back up for another 2 minutes then goes back to not responding.

I looked at the logs but saw nothing glaring but that does not say much -- I am not well versed at looking at the logs and I may not be looking at the correct ones.  Input on what to look for would be appreciated.  I next updated vCenter to 4.1 U1 and get the same results.  Next thing I tried was to add the existing server (running ESXi 4.0) and it behaves the same way.

These 2 servers I am trying to add are in a different network than my other 6 ESXi servers but are in the same physical location.  The vCenter box is in the same network as my functioning 6 hosts.  I had my network guy open up everythng between the 2 ESXi servers I am trying to add and the vCenter box and still the same result, although I still suspect a network issue.  I know connectivity is there because I can add the host and ping both ways.

Here is a run down of what is different on the 2 new hosts that I am trying to add:  in different network segment than vCenter, are ESXi embedded (other 6 have local hard disks,) and the licenses are different levels (although I can't imagine that has anything to do with it.)

Any ideas or help would be most appreciated.

[a_p_]

Some thoughts:

Are the hosts able to resolve the vCenter Server's FQDN?

Are there any hints in the host's vpxa log files?

André

[hbtadmin]

Nope... could not resolve the FQDN of the vCenter box.  Could not ping the IP address either, nor anything else in that subnet.  I went back to my network guy and he said it was wide open -- he even showed me.  I am still thinking there is something fishy with the network configuration, but it looks like it should work.

[a_p_]

Please double check the host's network settings and make sure there's no typo (e.g. in the gateway address) which causes the issue.

André

EDIT: Did you run the "Test Management Network" from the host's DCUI yet.

[Los · Accepted solution]

From the installation guide:

902 The default port that the vCenter Server system uses to send data to managed hosts. Managed hosts

also send a regular heartbeat over UDP port 902 to the vCenter Server system. This port must not be

blocked by firewalls between the server and the hosts, or between hosts.

If this port isn't open over UDP you see the exact thing you are experiencing. Give that a shot.

[hbtadmin]

OK... finally got it figured out.  Found a network security device inline (that I was unaware of) that was dropping the traffic between my host and vcenter.  Once I had that resolved I was able to ping but still dropped after 2 minutes.  Opened up 902 UDP and all works as it should now.  Thank you to everyone for your help.