Registered vCenter Marked "(unusable) <vcenter name>" in inventory

[pwmiller]

Good morning all,

I registered a vCenter Server in vCO 5.5 1281930 and it shows up in inventory with "(unusable) fqdn_of_vcenter". Since I registered this as a vCAC advanced services endpoint, I should only make changes from vCAC, but I cannot change or delete the endpoint as vCO throws the following error:

Error

Unable to create a vCO endpoint of type VC. Reason: Inconsistency between registered hosts and loaded hosts

Do you have any ideas on how to resolve this?

Thanks,

[pwmiller]

Some additional background and logs:

In the Configuration, the vCenter shows:

  Exception, 'org.apache.http.NoHttpResponseException: The target server failed to respond'

We are able to query the server from vCO over port 443, it just looks like the SDK is not responding... Any ideas?

In the Logs, we see:

2014-04-16 10:45:17.753-0400 [VimSessionManager validate sessions timer] ERROR {} [VimSession] login() [vCO@https://<serverfqdn>:443/sdk/Main#7f7e8129 useIS: true] -->

com.vmware.vim.vmomi.client.exception.TransportProtocolException: org.apache.http.NoHttpResponseException: The target server failed to respond

at com.vmware.vim.vmomi.client.common.impl.ResponseImpl.setError(ResponseImpl.java:224)

at com.vmware.vim.vmomi.client.http.impl.HttpExchange.run(HttpExchange.java:131)

at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingImpl.send(HttpProtocolBindingImpl.java:98)

at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl$CallExecutor.sendCall(MethodInvocationHandlerImpl.java:533)

at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl$CallExecutor.executeCall(MethodInvocationHandlerImpl.java:514)

at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.completeCall(MethodInvocationHandlerImpl.java:302)

at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.invokeFetch(MethodInvocationHandlerImpl.java:294)

at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.invoke(MethodInvocationHandlerImpl.java:165)

at com.sun.proxy.$Proxy240.getContent(Unknown Source)

at com.vmware.vmo.plugin.vi4.config.VlsiHelper.getServiceInstanceContent(VlsiHelper.java:234)

at com.vmware.vmo.plugin.vi4.config.VlsiHelper.login(VlsiHelper.java:182)

at com.vmware.vmo.plugin.vi4.config.VlsiHelper.loginWithUsername(VlsiHelper.java:222)

at com.vmware.vmo.plugin.vi4.VimSession._login(VimSession.java:414)

at com.vmware.vmo.plugin.vi4.VimSession.login(VimSession.java:381)

at com.vmware.vmo.plugin.vi4.VimSession.getServiceContent(VimSession.java:587)

at com.vmware.vmo.plugin.vi4.VimSession.getRootFolder(VimSession.java:1848)

at com.vmware.vmo.plugin.vi4.VimHost.checkUsable(VimHost.java:595)

at com.vmware.vmo.plugin.vi4.VimSession.pingSession(VimSession.java:2309)

at com.vmware.vmo.plugin.vi4.VimSessionManager$1.run(VimSessionManager.java:45)

at java.util.TimerThread.mainLoop(Unknown Source)

at java.util.TimerThread.run(Unknown Source)

Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond

at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:95)

at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)

at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)

at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)

at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)

at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:219)

at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)

at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)

at org.apache.http.impl.client.DefaultRequestDirector.createTunnelToTarget(DefaultRequestDirector.java:904)

at org.apache.http.impl.client.DefaultRequestDirector.establishRoute(DefaultRequestDirector.java:823)

at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:649)

at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)

at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)

at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)

at com.vmware.vim.vmomi.client.http.impl.HttpExchange.run(HttpExchange.java:111)

... 19 more

2014-04-16 10:45:17.754-0400 [VimSessionManager validate sessions timer] WARN  {} [VimSession] <serverfqdn> not usable

[lenny5348]

pwmiller, any chance you found a solution for this?  I have the same setup (vCAC 6.0.1 with built-in vCO and vCenter 5.5) and I'm experiencing the same scenario.  I checked the logs on the vCenter side and I'm capturing the following SSL error whenever I force the vCO to connect to vCenter:

[06512 warning 'ProxySvc'] SSL Handshake failed for stream <io_obj p:0x000000000f013118, h:3592, <TCP 'vCenter:443'>, <TCP 'vCAC:45566'>>, error: class Vmacore::Ssl::SSLException(SSL Exception: error:1407609B:SSL routines:SSL23_GET_CLIENT_HELLO:https proxy request)

I've tried deleting the existing endpoint from vCAC so that I can recreate it (Advanced Services > Endpoints) but I get this error when I try deleting it:

Unable to create a vCO endpoint of type VC. Reason: Inconsistency between registered hosts and loaded hosts

I found that I can manually add the vCenter host to vCO via the vCO configuration interface and this succeeds without error.  Meaning, I don't see any errors on the vCO side and none on the vCenter side.  However, when viewing the connection from the vCO java client, it shows invalid credentials.  Additionally, the new endpoint created from within vCO does not show in vCAC (Advanced Services > Endpoints).

This is the only thread I could find on this issue so hopefully we get some response.  Thanks in advance to anyone who can shine some light on this problem.

[pwmiller · Accepted solution]

Hey lenny5348,

I was able to resolve this issue, but probably not in the same way as you may have to. There was trouble with the vCenter Connection that was due to some arcane firewall rules in our environment - vCO responds to this by renaming the object in inventory. This of course causes the reference to that object (which is done by name) to become invalidated in vCAC (it now points to something that doesn't exist!).

We were able to remove the reference by modifying the name of the vCenter Endpoint on our vPostgres database to match what it shows in vCO. Unfortunately I don't have the SQL that I ran to fix this - but this might point you in the right direction.

VMware - if the vCO plugin renames the object when inaccessible, that means that vCAC can never cleanly remove inaccessible vCO endpoints through ASD. Perhaps the linkage should be made by some sort of UUID rather than by name?

[lenny5348]

Thank you for you reply, this is good information to know and is a big help!  Much appreciated!

[marsherian]

Here is some SQL that will update as needed.

WITH changes AS ( select id as sdk_id from asd_endpoint ae where rootobjectclass='VC:SdkConnection') update asd_endpoint AS e SET rootobjecturi='https://<URI_TEXT>:443/sdk' FROM changes ae where ae.sdk_id=e.id;

[marsherian]

Another quick note. The username and password used for the vCO Registration to the vCenter SDK URL must be the same as the ASD Endpoint.

I ran into an issue where I was using two different service accounts and new vCenter registrations were occuring in the vCO Instance on the vCAC server.

[kallischlauch]

top marks for this one 🙂 i added and failed 100 times, then in vRA i updated the settings for the endpoint and job done!