VC 2.5 losing connection to hosts

[tejama]

I've recently installed VC 2.5 and I've added our two 3.0.1 hosts.

The problem I'm having is that after about a minute and a half, I'm

losing connection to both hosts. Could anyone suggest a possible

solution to this issue?

Thanks.

[admin]

Did you do a fresh VC install or upgrade?

Do the servers reconnect automatically?

Did you replace the SSL certs on either VC or ESX?

At the time of the disconnects what do the VC logs, VC server event logs (application,system,security) and ESX logs show?

VC logs: C:\Documents & Settings\All Users\Application Data\VMware\VMware VirtualCenter\Logs

ESX logs: /var/log/vmware/hostd.log and /var/log/vmware/vpx/vpxa.log

[kjb007]

Did you connect the esx hosts via ip or hostname? Make sure to use hostname, and to verify that name resolution is working correctly. If you use DNS, make sure all entries in DNS correllate to the correct IP addresses. If not, make sure your hosts files for esx (/etc/hosts) and your vc server (c:\windows\system32\drivers\etc\hosts) have the correct info.

-KjB

[tejama]

I opened this question and was perhaps not perfectly clear. The issue I'm having is that the two ESX 3.0.1 hosts get greyed out after a period of about 70 seconds and are listed as "Not responding". I can disconnect the host via VC, Connect the host again and then use it properly for another 70 seconds before losing connection again.

I removed the vpxa package on the host and added it again, but got the same result. Any further suggestions?

[jayolsen]

Still need to verify dns or hosts files which ever you are using. Make FQDN resolution works, host.domain.com

Also make sure you don't have any other devices on your network with the same IPs as your ESX hosts.

[kjb007]

Are you using IP or FQDN when you registered the esx host? If you are using IP, change to hostname.

-KjB

[admin]

Is UDP port 902 open between the ESX servers and VC? Run wireshark on the VC server to check if it's receiving the packets.

If VC cannot receive the ESX heartbeat packet it will disconnect the ESX server.

Are there any errors in the VC logs?

[admin]

Also, there is a known issue, with the same symptoms you are having, for some ESX 3.0.x servers which have certain patches installed.

See the following KB: http://kb.vmware.com/kb/1004137/

[patrickds]

This was asked before, but you never answered:

is this a VC fresh install or an upgrade where you kept the database?

I have experienced these same issues after doing an upgrade to VC 2.5 on SQL2005 Express from an MSDE DB.

A fresh reinstall with a new db solved it.

There are procedures and agents in the older db that don't work on a 2005 Express db which caused the issues (simply put, more complete explanation can be found in the forums)

[tejama]

Thanks for all your suggestions. Unfortunately I'm still having the same issue that I originally posted. I'll provide some additional details below:

The VC server is seperated from the hosts ESX servers by a firewall. The Virtual Infrastructure client has been installed on this server for quite some time and has been working properly. When I connect to my hosts, I see using Tcpview that a connection on port 902 is established. So this rules out port 902 being blocked. I should also note that I do not get the "Not responding" error when using the basic VIC.

As a test, I installed VC using the exact same configuration, database, and license file on a server in the same zone as the ESX hosts. I didn't encounter any issues at all and it is not prone to the same "Not responding" error. I guess this rules out issues with the vpxa agent on the hosts.

@patrickds - This is a fresh install.

Can anybody provide some additional suggestions?

[admin]

You have confirmed that TCP 902 is open but is UDP 902 open?

VirtualCenter needs to receive UDP packets from the ESX servers on port 902.

If it does not, it will disconnect the hosts after about a minute.

Does Tcpview or wireshark show regular UDP 902 packets?

[tejama]

@appk - Through Tcpview I do see UDP packets on port 902. Thanks for the suggestion...

[patrickds]

You could try checking the firewall logs for dropped packets.

The fact that an identical installation in the same network has no issues seems to indicate that the problem lies somewhere in the network.

Windows firewall is not enabled on your VC by any chance?

[tejama]

Thanks Patrick. I'm going to mark this question as closed as I'm fairly confident this is turning out to be a firewall/network issue. Thanks to everyone for taking the time to answer