Vcenter web interface : 503 Service Unavailable

[Looow]

Hello, 

I use the vcenter appliance 6.7.

After a restart when I try to access to my Vcenter interface I get this error : 

" 503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http20NamedPipeServiceSpecE:0x00007f131c008930] _serverNamespace = / action = Allow _pipeName =/var/run/vmware/vpxd-webserver-pipe)"

When I try to start manualy all services I get this error:

"Service-control failed. Error: Failed to start services in profile ALL. RC=1, stderr=Failed to start vapi-endpoint, vpxd-svcs services. Error: Operation timed out" 

Disk usage is good, all disk have available space ...

Can you help me please ? 

[We11itDepends]

You can check on the certificates with the below commands:

Appliance:
for i in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list); do echo $i; /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $i --text | grep -i "not after"; done;

Check also STS certificate:
* Checking Expiration of STS Certificate on vCenter Server (79248)
https://kb.vmware.com/s/article/79248

If there are any expired certificates, renew them in this order:
1)Take offline snapshots of all nodes in the SSO domain
2) STS
3) The other certs.

To renew STS:
Appliance:
* "Signing certificate is not valid" - Regenerating and replacing expired STS certificate using shell script on vCenter Server Appliance 6.5/6.7 (76719)
https://kb.vmware.com/s/article/76719

To renew the other certs:
* How to regenerate vSphere 6.x certificates using self-signed VMCA (2112283)
https://kb.vmware.com/s/article/2112283

[Looow]

Thanks for your reply,

All certificates are not expired : 

"MACHINE_SSL_CERT
Not After : Oct 5 17:30:59 2031 GMT
TRUSTED_ROOTS
Not After : May 8 12:06:14 2029 GMT
Not After : Oct 5 17:30:59 2031 GMT
TRUSTED_ROOT_CRLS
machine
Not After : May 8 12:06:14 2029 GMT
vsphere-webclient
Not After : May 8 12:06:14 2029 GMT
vpxd
Not After : May 8 12:06:14 2029 GMT
vpxd-extension
Not After : May 8 12:06:14 2029 GMT
APPLMGMT_PASSWORD
data-encipherment
Not After : May 8 12:06:14 2029 GMT
SMS
Not After : May 14 12:10:36 2029 GMT
BACKUP_STORE
Not After : Oct 5 17:30:59 2031 GMT
Not After : May 8 12:06:14 2029 GMT
Not After : May 8 12:06:14 2029 GMT
Not After : May 8 12:06:14 2029 GMT
Not After : May 8 12:06:14 2029 GMT"

STS certificate are good too :

2 VALID CERTS
================

LEAF CERTS:

[] Certificate 73:B4:3F:1E:9C:C7:0A:74:88:87:6E:57:D3:8D:47:47:F8:58:2B:93 will expire in 2765 days (8 years).

ROOT CERTS:

[] Certificate 4E:A5:B2:C4:71:4A:7F:75:1B:96:EF:EC:0C:AA:42:37:39:98:08:EE will expire in 2765 days (8 years).

0 EXPIRED CERTS
================

LEAF CERTS:

None

ROOT CERTS:

None

I try to renewall certificates and I get errors :

"Don"t update service ...."

2021-10-12T08:45:03.435Z ERROR certificate-manager 'lstool get' failed: 1
2021-10-12T08:45:03.435Z ERROR certificate-manager 'lstool get' failed: 1
2021-10-12T08:45:03.435Z ERROR certificate-manager please see /var/log/vmware/vmcad/certificate-manager.log

Attached you can find the output of the reset certificates 

Any idea ? 

[shameerpp]

Check all the services are running, also restart services through SSH. find below article for more info

https://www.nakivo.com/blog/503-service-unavailable-error-on-the-vsphere-web-client/ 

[Ajay1988]

Failed to start vapi-endpoint, vpxd-svcs services. Error: Operation timed out" .

Did you try a certificate replacement which failed ? 

Run this cmd and share output >> /usr/lib/vmware-vmafd/bin/vmafd-cli get-pnid --server-name localhost

Please check /var/log/vmware/vpxd-svcs/vpxd-svcs.log and /var/log/vmware/vapi/endpoint/endpoint.log for errors. Please try to start all services (service-control --start --all) and after failure share those logs.