Can not add host to VC

[sfonten]

when i try to add a host to my VC i get "Unable to access the specified host. It either does not exist, the server software is not responding, or there is a network problem." i have tried to do a service mgmt-vmware restart and no luck

[e12pilot]

Is this a remote ESX server? Remote meaning separated by a WAN link to your VC server?

We are seeing the same problem over WAN links to ESX 2.5.4 hosts, no solution yet.

[hicksj]

Check DNS. Can each system resolve the other's FQDN?

[spina]

I'm having the same problem. I have just installed a new SSL certificate and I keep getting the same error msg. I've asked around at work and a few people say that the VC server is not accepting certificates from my ESX servers. I have navigated to the web page of the ESX server and downloaded it's cert, but still nothing works. Here is a copy of my log file with the error I am getting:

\[2007-09-18 10:48:07.158 'App' 2664 info] \[VpxLRO] -- BEGIN task-internal-457 -- datacenter-21 -- vim.Datacenter.queryConnectionInfo

\[2007-09-18 10:48:08.267 'App' 2664 error] \[VpxVmdbCnx] Authd error: Cannot connect to host losao225: No connection could be made because the target machine actively refused it

\[2007-09-18 10:48:08.267 'App' 2664 error] \[VpxVmdbCnx] Failed to connect to host losao225:902. Check that authd is running correctly (lib/connect error 2)

\[2007-09-18 10:48:09.345 'App' 2664 error] \[VpxVmdbCnx] Authd error: Cannot connect to host losao225: No connection could be made because the target machine actively refused it

\[2007-09-18 10:48:09.345 'App' 2664 error] \[VpxVmdbCnx] Failed to connect to host losao225:902. Check that authd is running correctly (lib/connect error 2)

\[2007-09-18 10:48:11.720 'BaseLibs' 2664 warning] SSLVerifyCertAgainstSystemStore: The remote host certificate has these problems:

\* A certificate in the host's chain is based on an untrusted root.

\[2007-09-18 10:48:11.720 'BaseLibs' 2664 warning] SSL: Unknown SSL Error

\[2007-09-18 10:48:11.720 'BaseLibs' 2664 warning] SSL Error: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

\[2007-09-18 10:48:11.720 'BaseLibs' 2664 warning] SSL: connect failed

\[2007-09-18 10:48:11.720 'App' 2664 error] \[VpxVmdbCnx] Authd error: The remote host certificate has these problems:

\* A certificate in the host's chain is based on an untrusted root.

\[2007-09-18 10:48:11.720 'App' 2664 error] \[VpxVmdbCnx] Failed to connect to host losao225:902. Check that authd is running correctly (lib/connect error 2)

\[2007-09-18 10:48:11.720 'App' 2664 info] \[VpxLRO] -- FINISH task-internal-457 -- datacenter-21 -- vim.Datacenter.queryConnectionInfo

\[2007-09-18 10:48:15.595 'SoapAdapter' 892 error] SOAP session count limit reached

\[2007-09-18 10:48:16.376 'SoapAdapter' 1928 error] SOAP session count limit reached

\[2007-09-18 10:48:25.079 'SoapAdapter' 892 error] SOAP session count limit reached

[e12pilot]

Which log file are you pulling this from? I would like to see if we are having the same problem.

Thanks!

[spina]

\WINDOWS\Temp\vpx\vpxd-X.log (X=0-9)

Also checked DNS everything is working fine

Message was edited by:

spina

[e12pilot]

Cool log file. I am seeing those certificate warnings, however since certificate verification is disabled, I don't believe that is the cause of our problems. I think this helps:

\[2007-09-18 10:06:15.070 'App' 3924 error] \[VpxVmdbCnx] Authd error: 221 Goodbye

\[2007-09-18 10:06:15.070 'App' 3924 error] \[VpxVmdbCnx] Failed to connect to host xxxxxxxxxx:902. Check that authd is running correctly (lib/connect error 11)

Doing a:

ps ax | grep -i authd[/code]

yields no results.

The strange thing is that when we go to add the host, the window pops up showing the host's configured portgroups, vmfs volumes, etc...so communication is occurring. The process fails when you click finish to actually add the host into the inventory.

Strange.

[sfonten]

i just rebooted the host and it worked fine. but t hat is just a quick fix

[johnswb]

I rebooted one of our host and that did the trick. Unfortunately I am unable to bounce all of them. Does anyone know if doing the following will do the trick, and will the VM's stay up?

"service mgmt-vmware restart"

If I remember correctly there was a file on the host that could be deleted and when add back to VC it would be replace. any ideas?

[e12pilot]

We had a case open with VMWare on this, and their only solution to the problem was to reboot the server.

[johnswb]

That is not good at all. We have several VM's that are high priority and can not be rebooted around xmas time. grrrr

[davegrant]

John, you should not generally need to reboot. "service mgmt-vmware restart" should do the trick, and if you are running 3.0.2 or 3.0.1 with patch ESX-7557441 this will not restart any VMs or affect running VMs in any way. If you are running 3.0.0 or 3.0.1 without that patch, just disable Automatic Startup under Virtual Machine Startup/Shutdown Config for that host first. You can also create the /tmp/vmware-root directory by hand as you alluded to, or install patch ESX-1002088 to prevent this issue from coming up in the first place.

Lots more info over at kb.vmware.com; try searching for "vpxa" or "virtualcenter agent."

-Dave

[admin]

Restarting the mgmt-vmware service will not shutdown the vms running on the host. Can you connect to the host directly with the VI client?

[e12pilot]

Just an FYI, I am referring to our 2.5.4 ESX hosts. VMware's solution for these servers was to reboot them. Restarting vmware-serverd or re-installing the virtual center service did not fix the problem for us. Only a cold reboot fixed the issue, our case was open for over a month, and trust me we were not happy with the solution either.

Not sure if this was fixed in a later 2.5.4 patch, or 2.5.5. We were running 2.5.4 Patch 4 with this issue, and are slowly trying to upgrade our hosts to 2.5.5.

[johnswb]

The wild thing is I have one ESX 2.5.3 and one 3.0.1 that are doing the same thing. I may have to wait until xmas is over before I can reconnect the host back to VC. I will restart mgmt-vmware to see it that does the trick.

[e12pilot]

Here is the other strange part of this....

This only occurs with our 2.5.4 hosts that are in remote datacenters in relation to the VC 2.x server. The ESX hosts (either 2.x, 3.x) that reside in close proximity (<5ms delay) to the virtual center server have never had a single issue. Our remote ESX servers are anywhere between 15ms and 230ms away from the VC host, and the one that is 230ms has never disconnected. It really is some strange behavior.

John, I am curious to see if your hosts are remote in proximity to your VC server, or if they are on the same LAN.

[admin]

Can you enable authd logging on the host? Add these lines to the /etc/vmare/config file on the host:

vmauthd.logEnabled = "TRUE"

log.vmauthdFileName = "/var/log/vmware/vmauthd.log"

Then try add the host again and check this log for any errors.

[johnswb]

e12pilot, our VC is in Arkansas and we have hosts in Arlington, Florida, Ft. Worth, Phoenix and St.Louis; none of them had any problems connecting to VC after the upgrade. Just two host here locally.

Also, restarting the mgmt-vmware did the trick for my ESX 3.0.1 server.

Thanks