503 Service unavailable. Already regenerated all certs.

[SirSully1]

vSphere vCenter 6.5

503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http20NamedPipeServiceSpecE:0x0000562ed04268f0] _serverNamespace = / action = Allow _pipeName =/var/run/vmware/vpxd-webserver-pipe)

Already regenerated all the certs and I'm still getting the above error and 

Service-control failed. Error Failed to start vmon services.vmon-cli RC=1, stderr=Failed to start sca, cm, vpxd-svcs, vapi-endpoint services. Error: Operation timed out. I have already gone through all the KB articles and I still can't get the web portal working.

[SirSully1]

root@vsphere [ /tmp ]# for i in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list); do echo STORE $i; sudo /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $i --text | egrep "Alias|Not After"; done
STORE MACHINE_SSL_CERT
Alias : __MACHINE_CERT
Not After : May 10 17:52:34 2025 GMT
STORE TRUSTED_ROOTS
Alias : ca58fb1b1b4fdf57eee20bd668d43bc4dae788ea
Not After : Nov 22 14:39:41 2028 GMT
Alias : 91c01a060afc5c23df427cfe8e1458f1a4aefdb2
Not After : Nov 25 18:52:13 2030 GMT
Alias : 524a8ae0d8d2a288a5956f001c662e9689a44d4d
Not After : Nov 25 19:14:14 2030 GMT
Alias : bab6c26305331c428c9e78f541e48fb10c4f4b12
Not After : Nov 25 16:02:52 2030 GMT
Alias : 901a13badaefe40467335f10bdb9903f0bde6700
Not After : Nov 25 16:46:57 2030 GMT
Alias : 67129e5adc4f7b1825667e28ca69c2c2750a2e55
Not After : Nov 26 11:40:07 2030 GMT
Alias : 26368e494047b8cccb1b2750e8ade71674af2c4c
Not After : Nov 26 16:20:39 2030 GMT
Alias : ca0ec9ba7da7d45802bbe34facfafa4b8f5a99af
Not After : May 5 13:59:34 2033 GMT
Alias : 88ab2b7b65fce7f7722234730156831174b85372
Not After : May 5 14:29:32 2033 GMT
Alias : 241e484d5d87bdd9b707d86c3b90ba9a9776b6fc
Not After : May 5 15:52:34 2033 GMT
Alias : 0a0484ad462539ad17a6d08ca820b15f2c6f5493
Not After : May 5 17:15:27 2033 GMT
Alias : 210417d9babe0e9bb14fd731af34b596589ee4ee
Not After : May 5 18:02:33 2033 GMT
STORE machine
Alias : machine
Not After : May 10 17:53:16 2025 GMT
STORE vsphere-webclient
Alias : vsphere-webclient
Not After : May 10 17:53:17 2025 GMT
STORE vpxd
Alias : vpxd
Not After : May 10 17:53:18 2025 GMT
STORE vpxd-extension
Alias : vpxd-extension
Not After : May 10 17:53:19 2025 GMT
STORE SMS
Alias : sms_self_signed
Not After : Nov 28 14:47:08 2028 GMT
STORE BACKUP_STORE

[LisandrodeCuba]

Had same symptoms long time ago in the end was not certificate related. Solved following this link. Take a look maybe this is also the case for you.

https://kb.vmware.com/s/article/2149278

[SirSully1]

thank you for the replay, but that did not fix the issue. I don't have huge log files.

root@vsphere [ /var/log/audit ]# ls -lh
total 660K
-rw------- 1 root root 4.4K May 12 13:59 audit.log
-r-------- 1 root root 260K May 12 00:00 audit.log.1
-r-------- 1 root root 132K May 11 03:20 audit.log.2
-r-------- 1 root root 120K May 10 03:00 audit.log.3
-r-------- 1 root root 123K May 9 03:12 audit.log.4

I still ran the truncate command but no love.

[hirschinho]

This KB helped me in the past:

https://kb.vmware.com/s/article/50113962