VMware Cloud Community
mspruit
Contributor
Contributor
Jump to solution

vCenter service won't start after renewing SSL

After renewing the SSL certificate on vCenter 7 with the Certificate Manager on VCSA the vSphere webclient doesn't start anymore. 

We received the error below:

2022-11-24T09:44:03.172Z  Updating certificate for "com.vmware.imagebuilder" extension

Reset status : 85% Completed [starting services...]

Error while starting services, please see service-control log for more details
Status : 0% Completed [Reset operation failed]

please see /var/log/vmware/vmcad/certificate-manager.log for more information.

 

In th evpxd.log file we've got the following error:

--> [context]zKq7AVECAQAAADx5MwEVdnB4ZAAAnuk3bGlidm1hY29yZS5zbwAAV4ksANV8LQC7+TIBibdvdnB4ZAABBXZ4AcN2eAGcDYIBjCHXAewZ1wHexdUCHCwBbGliYXV0aHpjbGllbnQuc28AAuM4AQJp4AECJ9UBAfyz1QE/228BG0xwAZ2jbwOHKwJsaWJjLnNvLjYAAYGZbw==[/context]>
2022-11-25T07:18:26.576Z info vpxd[02644] [Originator@6876 sub=AuthzStorageProvider] [AuthzStorageProvider::CreateAuthzMgr] Retry for this error: attempt count 60
2022-11-25T07:18:29.612Z info vpxd[02644] [Originator@6876 sub=vpxCrypt] Failed to read X509 cert; err: 151441516
2022-11-25T07:18:29.645Z info vpxd[02644] [Originator@6876 sub=SsoClient] Successfully acquired token: SamlToken [subject={Name: vpxd-9cb36f20-29fa-11eb-88d7-000c29e17dfb; Domain:vsphere.local}, groups=[{Name: Users; Domain:vsphere.local}, {Name: SolutionUsers; Domain:vsphere.local}, {Name: SystemConfiguration.Administrators; Domain:vsphere.local}, {Name: ComponentManager.Administrators; Domain:vsphere.local}, {Name: LicenseService.Administrators; Domain:vsphere.local}, {Name: ActAsUsers; Domain:vsphere.local}, {Name: Everyone; Domain:vsphere.local}], delegationChain=[], startTime=2022-11-25 07:18:29.613, expirationTime=2022-11-25 15:18:29.613, renewable=false, delegable=false, isSolution=true,confirmationType=1]
2022-11-25T07:18:29.650Z warning vpxd[02726] [Originator@6876 sub=IO.Connection] Failed to connect; <io_obj p:0x00007fc4e40d7408, h:32, <TCP '127.0.0.1 : 48850'>, <TCP '127.0.0.1 : 10080'>>, e: 111(Connection refused), duration: 0msec
2022-11-25T07:18:29.651Z warning vpxd[02726] [Originator@6876 sub=HttpConnectionPool-000071] Failed to get pooled connection; <cs p:00007fc50061f570, TCP:localhost:10080>, (null), duration: 0msec, N7Vmacore15SystemExceptionE(Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.)
--> [context]zKq7AVECAQAAADx5MwEPdnB4ZAAAnuk3bGlidm1hY29yZS5zbwAAV4ksANV8LQC7+TIAp9wjAEb4IwDTCCQACQ0kAPYUJABjuSMA51ojAI+wIwBcdTcBh38AbGlicHRocmVhZC5zby4wAAIPNg9saWJjLnNvLjYA[/context]
2022-11-25T07:18:29.652Z info vpxd[02726] [Originator@6876 sub=IO.Http] Set user agent error; state: 1, (null), N7Vmacore15SystemExceptionE(Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.)
--> [context]zKq7AVECAQAAADx5MwEPdnB4ZAAAnuk3bGlidm1hY29yZS5zbwAAV4ksANV8LQC7+TIAp9wjAEb4IwDTCCQACQ0kAPYUJABjuSMA51ojAI+wIwBcdTcBh38AbGlicHRocmVhZC5zby4wAAIPNg9saWJjLnNvLjYA[/context]
2022-11-25T07:18:29.654Z error vpxd[02726] [Originator@6876 sub=IO.Http] User agent failed to send request; (null), N7Vmacore15SystemExceptionE(Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.)
--> [context]zKq7AVECAQAAADx5MwEPdnB4ZAAAnuk3bGlidm1hY29yZS5zbwAAV4ksANV8LQC7+TIAp9wjAEb4IwDTCCQACQ0kAPYUJABjuSMA51ojAI+wIwBcdTcBh38AbGlicHRocmVhZC5zby4wAAIPNg9saWJjLnNvLjYA[/context]
2022-11-25T07:18:29.655Z warning vpxd[02644] [Originator@6876 sub=Authz] [ConnectAndLogin] Failed to loginBySamlToken: N7Vmacore15SystemExceptionE(Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.)
--> [context]zKq7AVECAQAAADx5MwEPdnB4ZAAAnuk3bGlidm1hY29yZS5zbwAAV4ksANV8LQC7+TIAp9wjAEb4IwDTCCQACQ0kAPYUJABjuSMA51ojAI+wIwBcdTcBh38AbGlicHRocmVhZC5zby4wAAIPNg9saWJjLnNvLjYA[/context]
2022-11-25T07:18:29.660Z info vpxd[02644] [Originator@6876 sub=Authz] fallback to loginByCertificate
2022-11-25T07:18:29.661Z warning vpxd[02737] [Originator@6876 sub=IO.Connection] Failed to connect; <io_obj p:0x00007fc52c009ad8, h:32, <TCP '127.0.0.1 : 48852'>, <TCP '127.0.0.1 : 10080'>>, e: 111(Connection refused), duration: 0msec
2022-11-25T07:18:29.661Z warning vpxd[02737] [Originator@6876 sub=HttpConnectionPool-000071] Failed to get pooled connection; <cs p:00007fc50061f570, TCP:localhost:10080>, (null), duration: 0msec, N7Vmacore15SystemExceptionE(Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.)
--> [context]zKq7AVECAQAAADx5MwEPdnB4ZAAAnuk3bGlidm1hY29yZS5zbwAAV4ksANV8LQC7+TIAp9wjAEb4IwDTCCQACQ0kAPYUJABjuSMA51ojAI+wIwBcdTcBh38AbGlicHRocmVhZC5zby4wAAIPNg9saWJjLnNvLjYA[/context]
2022-11-25T07:18:29.665Z info vpxd[02737] [Originator@6876 sub=IO.Http] Set user agent error; state: 1, (null), N7Vmacore15SystemExceptionE(Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.)
--> [context]zKq7AVECAQAAADx5MwEPdnB4ZAAAnuk3bGlidm1hY29yZS5zbwAAV4ksANV8LQC7+TIAp9wjAEb4IwDTCCQACQ0kAPYUJABjuSMA51ojAI+wIwBcdTcBh38AbGlicHRocmVhZC5zby4wAAIPNg9saWJjLnNvLjYA[/context]
2022-11-25T07:18:29.669Z error vpxd[02737] [Originator@6876 sub=IO.Http] User agent failed to send request; (null), N7Vmacore15SystemExceptionE(Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.)
--> [context]zKq7AVECAQAAADx5MwEPdnB4ZAAAnuk3bGlidm1hY29yZS5zbwAAV4ksANV8LQC7+TIAp9wjAEb4IwDTCCQACQ0kAPYUJABjuSMA51ojAI+wIwBcdTcBh38AbGlicHRocmVhZC5zby4wAAIPNg9saWJjLnNvLjYA[/context]
2022-11-25T07:18:29.669Z error vpxd[02644] [Originator@6876 sub=httpUtil] Error in sending request: N7Vmacore15SystemExceptionE(Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.)
--> [context]zKq7AVECAQAAADx5MwEPdnB4ZAAAnuk3bGlidm1hY29yZS5zbwAAV4ksANV8LQC7+TIAp9wjAEb4IwDTCCQACQ0kAPYUJABjuSMA51ojAI+wIwBcdTcBh38AbGlicHRocmVhZC5zby4wAAIPNg9saWJjLnNvLjYA[/context]
2022-11-25T07:18:29.672Z error vpxd[02644] [Originator@6876 sub=ServerAccess] Remote login failed: N3Vim5Fault9HttpFault9ExceptionE(Fault cause: vim.fault.HttpFault
--> )
--> [context]zKq7AVECAQAAADx5MwEVdnB4ZAAAnuk3bGlidm1hY29yZS5zbwAAV4ksANV8LQC7+TIBibdvdnB4ZAABvfuBgQyqYQEBIAuCAYwh1wHsGdcB3sXVAhwsAWxpYmF1dGh6Y2xpZW50LnNvAALjOAECaeABAifVAQH8s9UBP9tvARtMcAGdo28DhysCbGliYy5zby42AAGBmW8=[/context]
2022-11-25T07:18:29.674Z error vpxd[02644] [Originator@6876 sub=AuthzStorageProvider] [AuthzStorageProvider::CreateAuthzMgr] Failed to connect to Authz service: <N5Vmomi5Fault17HostCommunication9ExceptionE(Fault cause: vmodl.fault.HostCommunication
--> )
--> [context]zKq7AVECAQAAADx5MwEVdnB4ZAAAnuk3bGlidm1hY29yZS5zbwAAV4ksANV8LQC7+TIBibdvdnB4ZAABBXZ4AcN2eAGcDYIBjCHXAewZ1wHexdUCHCwBbGliYXV0aHpjbGllbnQuc28AAuM4AQJp4AECJ9UBAfyz1QE/228BG0xwAZ2jbwOHKwJsaWJjLnNvLjYAAYGZbw==[/context]>
2022-11-25T07:18:29.676Z error vpxd[02644] [Originator@6876 sub=Default] Failed to instantiate AuthzStorageProvider: N5Vmomi5Fault17HostCommunication9ExceptionE(Fault cause: vmodl.fault.HostCommunication
--> )
--> [context]zKq7AVECAQAAADx5MwEVdnB4ZAAAnuk3bGlidm1hY29yZS5zbwAAV4ksANV8LQC7+TIBibdvdnB4ZAABBXZ4AcN2eAGcDYIBjCHXAewZ1wHexdUCHCwBbGliYXV0aHpjbGllbnQuc28AAuM4AQJp4AECJ9UBAfyz1QE/228BG0xwAZ2jbwOHKwJsaWJjLnNvLjYAAYGZbw==[/context]
2022-11-25T07:18:29.678Z error vpxd[02644] [Originator@6876 sub=Authorize] Failed to initialize authorizeManager
2022-11-25T07:18:29.678Z warning vpxd[02644] [Originator@6876 sub=VpxProfiler] Start [VpxdAuthorize::Start()] took 185703 ms
2022-11-25T07:18:29.678Z warning vpxd[02644] [Originator@6876 sub=VpxProfiler] ServerApp::Start [TotalTime] took 185742 ms
2022-11-25T07:18:29.678Z error vpxd[02644] [Originator@6876 sub=Default] Failed to start VMware VirtualCenter. Shutting down
2022-11-25T07:18:29.678Z info vpxd[02644] [Originator@6876 sub=SupportMgr] Wrote uptime information
2022-11-25T07:18:51.946Z info vpxd[02776] [Originator@6876 sub=Health] Wrote vpxd health XML to file /var/run/vpxd/vmware-vpxd-health-status.xml. Status: START_PENDING. Expiration: 71719
2022-11-25T07:19:21.947Z info vpxd[02671] [Originator@6876 sub=Health] Wrote vpxd health XML to file /var/run/vpxd/vmware-vpxd-health-status.xml. Status: START_PENDING. Expiration: 71749
2022-11-25T07:19:21.949Z info vpxd[02644] [Originator@6876 sub=Default] Forcing shutdown of VMware VirtualCenter now

Is there anyone who can help me solve this issue? 

 

0 Kudos
1 Solution

Accepted Solutions
mspruit
Contributor
Contributor
Jump to solution

I've got VMware working again with help of lsdoctor tool.

The steps I followed are in the link below

https://vninja.net/2022/08/08/expired-vmware-vcenter-7-certificates/

View solution in original post

0 Kudos
2 Replies
maksym007
Expert
Expert
Jump to solution

I assume that your SSL was wrongly created and applied. 

Usually, after SSL Certificate replacement, there is no issues. 

To be calm - check also partition size. Who knows maybe you ran out of space. 

0 Kudos
mspruit
Contributor
Contributor
Jump to solution

I've got VMware working again with help of lsdoctor tool.

The steps I followed are in the link below

https://vninja.net/2022/08/08/expired-vmware-vcenter-7-certificates/

0 Kudos