Replace vCenter Server 5.5 SSL certs using vC-CAT fails

[MillardJK]

I've seen other threads related to 5.1 and failing to replace SSL certs on vCenter Server, but I'm not able to use any of those solutions to fix my problem.

This is a 2008R2 host that originally had 5.1 on it; that was a clean install, not an upgrade from 5.0 or earlier.

The CAT (Certificate Automation Tool) is able to cleanly replace & update trusts for SSO and the Inventory Service, but fails when trying to update vCenter Server. The failure appears similar to other problems posted--the symptom is a log entry that indicates there may be multiple certs, services or entries making things non-unique--but I've been able to validate that one (and only one) entry for vCenter Server exists, that everything is matching from a certificate perspective, and that the new certificates have the right attributes, key sizes, etc.

The one thing that is clear in this environment is the failed result from invoking the reloadSslCertificate method in the MOB (https://vcenter/mob/?moid=vpxd-securitymanager&vmodl=1) even before trying to do any certificate updates. The result reason for the failure is "data buffer too large". Not a lot more information is available in vpxd.log.

[Unix-Sysadmin]

Hi.

We have the same problem.

Do you have a solution for this problem?

[MillardJK]

I have not. And because it's my home lab, I'm not able to open a support case; at this point, I'm looking at rebuilding vCenter from scratch (after backing-out all my VDS configurations--meh).

[ekrejci]

Hi,

I'm also having this issue with the reloadSslCertificate method.

I've read that you want to reinstall your vcenter. just to let you know that it is after a reinstallation that I cannot change the SSL cert with a SSL that already changed in 5.1.

maybe a new issue with 5.5...

I'll open a SR to see why I cannot change this Cert.

let you informed.

Eric.

[kiddsmith]

I've been fighting the same problem(s) for the past week, attempting to replace my vCenter Server 5.5 certificates.  I've finally got it working, and want to share my finding for the benefit of others who come across this thread.

First, I believe the "data buffer too large" error is the result of connecting to localhost using Internet Explorer with Compatibility View disabled.  (Alternatively, it might be that localhost needs to be added to the Trusted Sites security zone-- I did both at the same time, but I've seen reports that an IE revision changed how it sends data to web sites, causing them to break.)

Second, enabling Compatibility View caused a second error, when attempting to Invoke Method:

Method Invocation Result: vpx.fault.SecurityConfigFault

I found the solution to this issue on Jack Stromberg's blog:

http://jackstromberg.com/2012/12/method-invocation-result-vpx-fault-securityconfigfault-when-replaci...

With the new certificates in place in the VMware VirtualCenter SSL director, reboot the server.  Restarting the Virtual Center service may be enough, but I opted for a full reboot.  Once the server has rebooted and the services started, go back to the MOB and try again.

Finally, there may be a difference between the manual replacement instructions in KB 2034833 and the process used by the Certificate Updater Tool.  The instructions for the tool call for concatenating the server and CA chain certificates, while the manual instructions keep the two sets apart and only combine them when generating the rui.pfx file.

While my certificate requests were generated by the Certificate Updater Tool, I opted to follow the knowledge base article's instructions to populate rui.crt only with the server certificate.  This might not make a difference, and now that I have my certificate in place, I'm not inclined to test further.