VMware Cloud Community
parisvc
Enthusiast
Enthusiast
Jump to solution

Update manager missing vcenter 6.5. vmware-updatemgr does not start.

 

I'm using vcenter 6.5.0.33000 and update manager is missing.

If I trying and start it from the console I get this:

service-control --start vmware-updatemgr

Perform start operation. vmon_profile=None, svc_names=['vmware-updatemgr'], include_coreossvcs=False, include_leafossvcs=False

2021-02-09T15:49:03.302Z   Service updatemgr state STOPPED

Successfully started service updatemgr

In /var/log/vmware/vmware-updatemgr/updatemgr-utility.logI see this:

[2021-02-09 15:40:55,949 ERROR] Unable to update CM service info

[2021-02-09 15:49:03,570 INFO] Install Key store for Jetty

[2021-02-09 15:49:05,017 INFO] Keystore installed successfully.

[2021-02-09 15:49:05,314 INFO] Updating VUM extension with VC

[2021-02-09 15:49:05,631 INFO] Updating CM service info

[2021-02-09 15:49:05,631 INFO] Enter updateCmServiceInfo

[2021-02-09 15:49:05,631 INFO] Client's property file: /usr/lib/vmware-updatemgr/firstboot/updatemgr.properties

[2021-02-09 15:49:05,714 ERROR] CM ReRegisterService failure. Exception is (cis.cm.fault.ComponentManagerFault) {

   dynamicType = <unset>,

   dynamicProperty = (vmodl.DynamicProperty) [],

   msg = '',

   faultCause = <unset>,

   faultMessage = (vmodl.LocalizableMessage) [],

   errorCode = 0,

   errorMessage = 'UNKNOWN'

}

[2021-02-09 15:49:05,715 ERROR] Unable to update CM service info

I've tried resetting the updatemgr-util db refreshing certs and vc.

But the problem persists.

0 Kudos
1 Solution

Accepted Solutions
parisvc
Enthusiast
Enthusiast
Jump to solution

Thanks. I did this but it didn't fix the issue but lead me on the right path.

I found this which resolved it.

https://www.reddit.com/r/vmware/comments/e4w74j/vsphere_67_u3a_vsan_html5_failed_to_extract/

and this post in particular:

Once the SSL trust anchors are mismatched, not amount of regenerating or replacing the Machine SSL certificate is going to fix it, it requires manual correction.

I have a script that I've used countless times to troubleshoot and correct this issue:

https://web.vmware-labs.com/scripts/check-trust-anchors

I recommend running it first with the following options:

./check-trust-anchors -cml

This will:

  1. colorize the output for quick identification of the pertinent pieces of information
  2. print the current Machine SSL certificates for all PSC/vCenters for comparison (assuming they're reachable on the network)
  3. denote that it is being run on a live system (as opposed to a support bundle), and will automatically dump the Lookup Service registrations

The script will print the unique SSL certificates being used as SSL trust anchors within the Lookup Service registrations. Each vCenter/PSC node should have certain services registered with the Lookup Service, and as such each node should have its current Machine SSL certificate listed (there are other solutions that would have one, Site Recovery Manager for instance). You can list the URIs that are using each unique certificate by including the -e option.

If you verify that your current Machine SSL certificate does not match the SSL certificate that the service registrations are using for the URIs that are pointed to that node, you can try and have the script fix them by including the -f option. This will list the unique certificates and then prompt you for four things:

  1. The SSO administrator account (defaults to "administrator@<your-sso-domain>", you can just hit Enter to accept the default)
  2. The SSO administrator password
  3. The SHA1 thumbprint of the certificate you want to replace (this should be one of the Endpoint certificates, not your current Machine SSL certificate)
  4. The FQDN/IP of the node you want to replace. The script needs to be run on one of the PSCs in the SSO domain. If your PSC is embedded, you can run it from the vCenter. If you are running it from the node you want to update the SSL trust anchors for, you can simply type "localhost"

The script uses the included /usr/lib/vmidentity/tools/scripts/ls_update_certs.py Python utility, which in turn uses the included Lookup Service libraries. You should see output similar to when you replace the Machine SSL certificate using the /usr/lib/vmware-vmca/bin/certificate-manager utility: at the end it should tell you how many services it updated (and it needs to be more than zero). If you do not receive a message stating how many services it updated, that means it hit a service registration it couldn't parse (as is a common problem with this utility), and you should run the check-trust-anchors script again to see if there are still any outstanding service registrations that are still using the wrong SSL certificate.

Usual caveats apply: take offline snapshots of all PSCs in the SSO domain before attempting to fix anything.

Edit: I just read the KB you linked, was your environment upgraded from 5.5? If so, there's some additional things you need to check (legacy SSO endpoints and a separate store in VECS).

 

 

View solution in original post

2 Replies
Ajay1988
Expert
Expert
Jump to solution

Have you tried this   https://kb.vmware.com/s/article/76298?lang=en_US 

If you think your queries have been answered
Mark this response as "Correct" or "Helpful".

Regards,
AJ
parisvc
Enthusiast
Enthusiast
Jump to solution

Thanks. I did this but it didn't fix the issue but lead me on the right path.

I found this which resolved it.

https://www.reddit.com/r/vmware/comments/e4w74j/vsphere_67_u3a_vsan_html5_failed_to_extract/

and this post in particular:

Once the SSL trust anchors are mismatched, not amount of regenerating or replacing the Machine SSL certificate is going to fix it, it requires manual correction.

I have a script that I've used countless times to troubleshoot and correct this issue:

https://web.vmware-labs.com/scripts/check-trust-anchors

I recommend running it first with the following options:

./check-trust-anchors -cml

This will:

  1. colorize the output for quick identification of the pertinent pieces of information
  2. print the current Machine SSL certificates for all PSC/vCenters for comparison (assuming they're reachable on the network)
  3. denote that it is being run on a live system (as opposed to a support bundle), and will automatically dump the Lookup Service registrations

The script will print the unique SSL certificates being used as SSL trust anchors within the Lookup Service registrations. Each vCenter/PSC node should have certain services registered with the Lookup Service, and as such each node should have its current Machine SSL certificate listed (there are other solutions that would have one, Site Recovery Manager for instance). You can list the URIs that are using each unique certificate by including the -e option.

If you verify that your current Machine SSL certificate does not match the SSL certificate that the service registrations are using for the URIs that are pointed to that node, you can try and have the script fix them by including the -f option. This will list the unique certificates and then prompt you for four things:

  1. The SSO administrator account (defaults to "administrator@<your-sso-domain>", you can just hit Enter to accept the default)
  2. The SSO administrator password
  3. The SHA1 thumbprint of the certificate you want to replace (this should be one of the Endpoint certificates, not your current Machine SSL certificate)
  4. The FQDN/IP of the node you want to replace. The script needs to be run on one of the PSCs in the SSO domain. If your PSC is embedded, you can run it from the vCenter. If you are running it from the node you want to update the SSL trust anchors for, you can simply type "localhost"

The script uses the included /usr/lib/vmidentity/tools/scripts/ls_update_certs.py Python utility, which in turn uses the included Lookup Service libraries. You should see output similar to when you replace the Machine SSL certificate using the /usr/lib/vmware-vmca/bin/certificate-manager utility: at the end it should tell you how many services it updated (and it needs to be more than zero). If you do not receive a message stating how many services it updated, that means it hit a service registration it couldn't parse (as is a common problem with this utility), and you should run the check-trust-anchors script again to see if there are still any outstanding service registrations that are still using the wrong SSL certificate.

Usual caveats apply: take offline snapshots of all PSCs in the SSO domain before attempting to fix anything.

Edit: I just read the KB you linked, was your environment upgraded from 5.5? If so, there's some additional things you need to check (legacy SSO endpoints and a separate store in VECS).