Strange error vCenter 5.5 u2 ssolscli.cmd OperationFailed listServices and updateServices

[dsheehan87]

Hi,

So here is my setup.  I recently migrated from vSphere 5.1 u1 to vSphere 5.5 u2 by way of building a new Windows 2012 R2 server, migrating the SSL directory from ProgramData, and creating and ODBC connection to the existing SQL 2012 DB (I did an upgrade of the DB as well).  This is try #2 - the first try was on a server that was named as the original, and then the old 5.1 server unjoined from the domain, this one joined etc.  In the end that was scrapped because as the time I had thought it was just something messed up with the install, the server etc.

This server was a fresh manual build of  Windows 2012 R2 (meaning I didn't deploy via a template etc).  Same issues.  The reason for upgrading this way is that I am running Citrix XenDesktop and it is a major hassle when using the MCS to allocate VM's to point to a new vSphere instance (it breaks it).

So, the install went perfect, and it was on to generating new certificates for the server, and having done this many times I tried it manual and also used the Derek Seaman vCenter 5.5 toolkit script to generate certs.  Then when moving into the Cert Automation tool, running ssl-updater.bat failed with java errors at the point where it should prompt for the admin password when replacing the SSO Certificate (same deal for inventory etc).  Why I cannot find a valid reason, so, I then moved on to a manual replacement.

To list the services:

ssolscli.cmd listServices https://blabla.local:7444/lookupservice/sdk

Intializing registration provider...

Getting SSL certificates for https://blabla.local:7444/lookupservice/sdk

General failure.

com.vmware.vim.sso.admin.exception.InternalError: General failure.

Return code is: OperationFailed

100

The thing is the Toolkit Script did create the sts gc and admin properties and _id files - how it acquired those numbers I have no idea though.

So I then tried to update the services - admin and gc worked!  Lastly I tried sts and (note thats not my weak password but my password currently does have a . and @ in it):

ssolscli.cmd updateService -d https://blabla.local:7444/lookupservice/sdk -u administrator@vsphere.local -p My@pass.w0rd -si c:\certs\vCenterSSO\sts_id -ip c:\certs\vCenterSSO\sts.properties

Intializing registration provider...

Getting SSL certificates for https://blabla.local:7444/lookupservice/sdk

General failure.

Return code is: ServiceNotResponding

2

Note I was at least able to change the web services cert so at least Xendesktop can see the cluster, but I need to get inventory, sso etc certificates replaced and I am at the end of my rope LOL

Any idea what could cause this would be appreciated.  Seeing that its the second server, one a custom windows template, and one a fresh manual setup, I doubt its anything build related.  I have found no one that has had these same issues (have been searching for 2 days!)

Thanks in advance!

NOTE: tried numerous other things like disabling ipv6, adding host entries, checking firewall logs, etc and the CA and Sub CA of these certs are trusted

[SeanWhitney]

Can you paste the ssoAdminServer.log and lookupservice.log file? It should be in c:\programdata\vmware\cis\runtime\vmwareSTS\logs.

[dsheehan87]

Another thing to note, is the even though the first two ssolscli updateService commands worked initially (admin and gc), I can not updateService on those anymore either.  This happened on both servers that I tried this, after the first attempt that was successful, it could no longer be updated.

I don't know why/how but I just have this suspicion that there is a connection between this and the errors on the Cert Automation tool just bombing out at the point where it should prompt for a password on updating/replacing a cert.

This is the error I get running the automation tool to update a cert:

Enter the Single Sign-On Administrator password (will not be echoed): Exception

in thread "main" java.lang.NoClassDefFoundError: com/sun/jna/platform/win32/WinN

T

        at java.lang.ClassLoader.defineClass1(Native Method)

        at java.lang.ClassLoader.defineClass(Unknown Source)

        at java.security.SecureClassLoader.defineClass(Unknown Source)

        at java.net.URLClassLoader.defineClass(Unknown Source)

        at java.net.URLClassLoader.access$100(Unknown Source)

        at java.net.URLClassLoader$1.run(Unknown Source)

        at java.net.URLClassLoader$1.run(Unknown Source)

        at java.security.AccessController.doPrivileged(Native Method)

        at java.net.URLClassLoader.findClass(Unknown Source)

        at java.lang.ClassLoader.loadClass(Unknown Source)

        at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)

        at java.lang.ClassLoader.loadClass(Unknown Source)

        at com.vmware.sso.cfg.AskPasswd.main(AskPasswd.java:46)

Caused by: java.lang.ClassNotFoundException: com.sun.jna.platform.win32.WinNT

        at java.net.URLClassLoader$1.run(Unknown Source)

        at java.net.URLClassLoader$1.run(Unknown Source)

        at java.security.AccessController.doPrivileged(Native Method)

        at java.net.URLClassLoader.findClass(Unknown Source)

        at java.lang.ClassLoader.loadClass(Unknown Source)

        at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)

        at java.lang.ClassLoader.loadClass(Unknown Source)

        ... 13 more

Exception in thread "main" java.lang.NoClassDefFoundError: groovy/lang/GroovyObj

ect

        at java.lang.ClassLoader.defineClass1(Native Method)

        at java.lang.ClassLoader.defineClass(Unknown Source)

        at java.security.SecureClassLoader.defineClass(Unknown Source)

        at java.net.URLClassLoader.defineClass(Unknown Source)

        at java.net.URLClassLoader.access$100(Unknown Source)

        at java.net.URLClassLoader$1.run(Unknown Source)

        at java.net.URLClassLoader$1.run(Unknown Source)

        at java.security.AccessController.doPrivileged(Native Method)

        at java.net.URLClassLoader.findClass(Unknown Source)

        at java.lang.ClassLoader.loadClass(Unknown Source)

        at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)

        at java.lang.ClassLoader.loadClass(Unknown Source)

        at sun.launcher.LauncherHelper.checkAndLoadMain(Unknown Source)

Caused by: java.lang.ClassNotFoundException: groovy.lang.GroovyObject

        at java.net.URLClassLoader$1.run(Unknown Source)

        at java.net.URLClassLoader$1.run(Unknown Source)

        at java.security.AccessController.doPrivileged(Native Method)

        at java.net.URLClassLoader.findClass(Unknown Source)

        at java.lang.ClassLoader.loadClass(Unknown Source)

        at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)

        at java.lang.ClassLoader.loadClass(Unknown Source)

        ... 13 more

[Fri 02/13/2015 -  9:00:27.85]: Last operation update Inventory Service SSL cert

ificate failed :

[Fri 02/13/2015 -  9:00:27.87]:

==================================================================

4. Update the Inventory Service SSL Certificate

[dsheehan87]

And it seems to not be only ssolscli.cmd that has issues...

C:\Program Files\VMware\Infrastructure\Inventory Service\scripts>unregister-sso.

bat https://blabla.local:7444/lookupservice/sdk administrator@vsphere.local password

Intializing registration provider...

Getting SSL certificates for https://blabla.local:7444/lookupservice/sdk

General failure.

Return code is: ServiceNotResponding

2

[dsheehan87 · Accepted solution]

OK - this is resolved, and when I tell you how, you will be astounded at the fix.  End result is I reinstalled everything on a new server.

In order to avoid this:

1) run a checksum on your ISO

2) verify the ISO checksum against the checksum on vmware.com

3) install if it matches - if not delete the ISO and go back to step 1

4) it works

What I did as it seems looking through my history, is that i ran a checksum on the ISO for ESXi, which checked out fine (I downloaded 8 ISO's etc from Microsoft and VMWare at the same time and left for lunch the other day so I was just going to checksum the files as I used them).  This is what I get for rushing (well in my defense LOL I was rushing because the 5.1 server was toast).

Whoever looked at this thread,thank you for your time!