1 2 Previous Next 15 Replies Latest reply on Jun 18, 2020 2:42 AM by ssSFrankSss

    Cannot start vCenter: 503 Service unavailable

    ssSFrankSss Novice

      Hi,

       

      I get this message from web browser, while I try to connect to vCenter:

      503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http20NamedPipeServiceSpecE:0x0000557c588763f0] _serverNamespace = / action = Allow _pipeName =/var/run/vmware/vpxd-webserver-pipe)

       

      And I have this image from shell that shows which services have started:

       

       

      There are like many solutions online, but I do not know where to start...

       

      I appreciate your time trying to help me,

      Frank

        • 2. Re: Cannot start vCenter: 503 Service unavailable
          ssSFrankSss Novice

          I hope this helps also in order to find a solution

           

          Command> service-control --status

          Running:

          applmgmt lwsmd vmafdd vmcad vmdird vmdnsd vmonapi vmware-cis-license vmware-eam vmware-psc-client vmware-rhttpproxy vmware-statsmonitor vmware-sts-idmd vmware-stsd vmware-vmon vmware-vpostgres vsphere-client vsphere-ui

          Stopped:

          pschealth vmcam vmware-cm vmware-content-library vmware-imagebuilder vmware-mbcs vmware-netdumper vmware-perfcharts vmware-rbd-watchdog vmware-sca vmware-sps vmware-updatemgr vmware-vapi-endpoint vmware-vcha vmware-vpxd vmware-vpxd-svcs vmware-vsan-health vmware-vsm

          Command> service-control --start --all

          Perform start operation. vmon_profile=ALL, svc_names=None, include_coreossvcs=True, include_leafossvcs=True

          2020-06-16T10:39:26.127Z   Running command: ['/usr/bin/systemctl', 'is-enabled', u'lwsmd']

          2020-06-16T10:39:26.136Z   Done running command

          2020-06-16T10:39:26.144Z   Service lwsmd does not seem to be registered with vMon. If this is unexpected please make sure your service config is a valid json. Also check vmon logs for warnings.

          2020-06-16T10:39:26.144Z   Running command: ['/sbin/service', u'lwsmd', 'status']

          2020-06-16T10:39:26.210Z   Done running command

          Successfully started service lwsmd

          2020-06-16T10:39:26.221Z   Running command: ['/usr/bin/systemctl', 'is-enabled', u'vmafdd']

          2020-06-16T10:39:26.236Z   Done running command

          2020-06-16T10:39:26.245Z   Service vmafdd does not seem to be registered with vMon. If this is unexpected please make sure your service config is a valid json. Also check vmon logs for warnings.

          2020-06-16T10:39:26.245Z   Running command: ['/sbin/service', u'vmafdd', 'status']

          2020-06-16T10:39:26.308Z   Done running command

          Successfully started service vmafdd

          2020-06-16T10:39:26.317Z   Running command: ['/usr/bin/systemctl', 'is-enabled', u'vmdird']

          2020-06-16T10:39:26.332Z   Done running command

          2020-06-16T10:39:26.341Z   Service vmdird does not seem to be registered with vMon. If this is unexpected please make sure your service config is a valid json. Also check vmon logs for warnings.

          2020-06-16T10:39:26.341Z   Running command: ['/sbin/service', u'vmdird', 'status']

          2020-06-16T10:39:26.404Z   Done running command

          Successfully started service vmdird

          2020-06-16T10:39:26.413Z   Running command: ['/usr/bin/systemctl', 'is-enabled', u'vmcad']

          2020-06-16T10:39:26.428Z   Done running command

          2020-06-16T10:39:26.436Z   Service vmcad does not seem to be registered with vMon. If this is unexpected please make sure your service config is a valid json. Also check vmon logs for warnings.

          2020-06-16T10:39:26.437Z   Running command: ['/sbin/service', u'vmcad', 'status']

          2020-06-16T10:39:26.500Z   Done running command

          Successfully started service vmcad

          2020-06-16T10:39:26.509Z   Running command: ['/usr/bin/systemctl', 'is-enabled', u'vmware-sts-idmd']

          2020-06-16T10:39:26.524Z   Done running command

          2020-06-16T10:39:26.533Z   Service vmware-sts-idmd does not seem to be registered with vMon. If this is unexpected please make sure your service config is a valid json. Also check vmon logs for warnings.

          2020-06-16T10:39:26.533Z   Running command: ['/sbin/service', u'vmware-sts-idmd', 'status']

          2020-06-16T10:39:26.595Z   Done running command

          Successfully started service vmware-sts-idmd

          2020-06-16T10:39:26.604Z   Running command: ['/usr/bin/systemctl', 'is-enabled', u'vmware-stsd']

          2020-06-16T10:39:26.618Z   Done running command

          2020-06-16T10:39:26.626Z   Service vmware-stsd does not seem to be registered with vMon. If this is unexpected please make sure your service config is a valid json. Also check vmon logs for warnings.

          2020-06-16T10:39:26.626Z   Running command: ['/sbin/service', u'vmware-stsd', 'status']

          2020-06-16T10:39:26.690Z   Done running command

          Successfully started service vmware-stsd

          2020-06-16T10:39:26.698Z   Running command: ['/usr/bin/systemctl', 'is-enabled', u'vmdnsd']

          2020-06-16T10:39:26.713Z   Done running command

          2020-06-16T10:39:26.722Z   Service vmdnsd does not seem to be registered with vMon. If this is unexpected please make sure your service config is a valid json. Also check vmon logs for warnings.

          2020-06-16T10:39:26.722Z   Running command: ['/sbin/service', u'vmdnsd', 'status']

          2020-06-16T10:39:26.784Z   Done running command

          Successfully started service vmdnsd

          2020-06-16T10:39:26.794Z   Running command: ['/usr/bin/systemctl', 'is-enabled', u'vmware-psc-client']

          2020-06-16T10:39:26.809Z   Done running command

          2020-06-16T10:39:26.818Z   Service vmware-psc-client does not seem to be registered with vMon. If this is unexpected please make sure your service config is a valid json. Also check vmon logs for warnings.

          2020-06-16T10:39:26.818Z   Running command: ['/sbin/service', u'vmware-psc-client', 'status']

          2020-06-16T10:39:26.880Z   Done running command

          Successfully started service vmware-psc-client

          Service-control failed. Error Failed to start vmon services.vmon-cli RC=2, stderr=Failed to start sca, cm services. Error: Service crashed while starting

           

          PS) vCenter runs as a VM

          • 3. Re: Cannot start vCenter: 503 Service unavailable
            daphnissov Guru
            vExpertCommunity Warriors

            Have you searched these forums for past threads and possible resolutions? This question comes up weekly with many possible causes.

            • 4. Re: Cannot start vCenter: 503 Service unavailable
              msripada Expert
              vExpert

              can you check the cm logs /var/log/vmware/cm/cm.log and post it here so that we can review.. cm service failed to start and thus other dependent services as well

               

              thanks,

              MS

              • 5. Re: Cannot start vCenter: 503 Service unavailable
                ssSFrankSss Novice

                I have tried some. But I was lost since there are many questions and solutions as you said!!

                • 6. Re: Cannot start vCenter: 503 Service unavailable
                  ssSFrankSss Novice

                  I have attached the log file thank you!

                  • 7. Re: Cannot start vCenter: 503 Service unavailable
                    Sukanyad Enthusiast
                    VMware Employees

                    From the logs i see that the cert expired :

                     

                    Caused by: java.security.cert.CertificateExpiredException: NotAfter: Wed May 06 19:32:13 UTC 2020

                            at sun.security.x509.CertificateValidity.valid(CertificateValidity.java:274) ~[?:1.8.0_162]

                            at sun.security.x509.X509CertImpl.checkValidity(X509CertImpl.java:629) ~[?:1.8.0_162]

                            at sun.security.provider.certpath.BasicChecker.verifyValidity(BasicChecker.java:190) ~[?:1.8.0_162]

                            at sun.security.provider.certpath.BasicChecker.check(BasicChecker.java:144) ~[?:1.8.0_162]

                            at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:125) ~[?:1.8.0_162]

                            at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:223) ~[?:1.8.0_162]

                     

                    Seems we might have any expired certs in VECS store .

                     

                    You can follow below kb to verify:

                    Kb2111411

                     

                    1 person found this helpful
                    • 8. Re: Cannot start vCenter: 503 Service unavailable
                      a.p. Guru
                      Community WarriorsvExpertUser Moderators

                      Please run the following commands from the vCSA's shell, to check whether one, or more certificates have expired.

                      /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store MACHINE_SSL_CERT --text

                      /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store machine --text

                      /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store vpxd --text

                      /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store vsphere-webclient

                      André

                      1 person found this helpful
                      • 9. Re: Cannot start vCenter: 503 Service unavailable
                        ssSFrankSss Novice

                        Thank you all it was a certificate issue this article helped me to return everything back to normal:

                         

                        VMware Knowledge Base

                        • 10. Re: Cannot start vCenter: 503 Service unavailable
                          toreshavn Novice

                          Hi Andres,

                           

                          How to run these commands from Window. Thanks !

                          • 11. Re: Cannot start vCenter: 503 Service unavailable
                            toreshavn Novice

                            Hi Andres,

                             

                            How to run these commands from Vcenter Windows.
                            If there is certificate expired, how to extend it.

                            I am having the same issues but with Vcenter Windows. Thank you !

                             

                             

                             

                            • 12. Re: Cannot start vCenter: 503 Service unavailable
                              ssSFrankSss Novice

                              toreshavn have you tried the article I have proposed, it has steps to follow for vCenter for windows also!

                              • 13. Re: Cannot start vCenter: 503 Service unavailable
                                toreshavn Novice

                                I have tried to the article you have proposed and reset all certificates, but it stops at 85% for starting services.
                                Please see image attached for the detail.

                                 

                                 

                                ============================================
                                Here is log:

                                2020-06-18T09:10:21.262Z INFO certificate-manager Serial number before replacement: f1:e8:d2:42:ea:06:14:b6

                                2020-06-18T09:10:21.262Z INFO certificate-manager Serial number after replacement: dc:17:ba:94:d5:9f:ef:7c

                                2020-06-18T09:10:21.262Z INFO certificate-manager Thumbprint before replacement: 46:2E:AD:D9:76:89:4D:60:4E:7B:EE:34:A0:E3:05:97:34:C8:70:55

                                2020-06-18T09:10:21.262Z INFO certificate-manager Thumbprint after replacement: B0:DA:23:46:E2:10:FB:6C:95:BA:30:DB:05:52:C9:55:8E:2C:41:C0

                                2020-06-18T09:10:21.263Z INFO certificate-manager vpxd-extension certificate replaced successfully. SerialNumber and Thumbprint changed.

                                2020-06-18T09:10:21.263Z INFO certificate-manager Running command :- "C:\Program Files\VMware\vCenter Server\bin\service-control.bat" --stop --ignore  --all

                                2020-06-18T09:10:21.263Z INFO certificate-manager please see service-control.log for service status

                                2020-06-18T09:11:22.345Z INFO certificate-manager Command executed successfully

                                2020-06-18T09:11:22.345Z INFO certificate-manager all services stopped successfully.

                                2020-06-18T09:11:22.345Z INFO certificate-manager None

                                2020-06-18T09:11:32.348Z INFO certificate-manager Running command :- "C:\Program Files\VMware\vCenter Server\bin\service-control.bat" --start  --all

                                2020-06-18T09:11:32.348Z INFO certificate-manager please see service-control.log for service status

                                2020-06-18T09:19:16.961Z ERROR certificate-manager None

                                2020-06-18T09:19:16.964Z ERROR certificate-manager Error while starting services, please see log for more details

                                2020-06-18T09:19:16.966Z ERROR certificate-manager {

                                    "resolution": null,

                                    "detail": [

                                        {

                                            "args": [

                                                "None"

                                            ],

                                            "id": "install.ciscommon.command.errinvoke",

                                            "localized": "An error occurred while invoking external command : 'None'",

                                            "translatable": "An error occurred while invoking external command : '%(0)s'"

                                        },

                                        "Error while starting services, please see log for more details"

                                    ],

                                    "componentKey": null,

                                    "problemId": null

                                • 14. Re: Cannot start vCenter: 503 Service unavailable
                                  ssSFrankSss Novice

                                  Actually I had the same problem... It stopped at 85% when starting services! Initially I thought it was a problem too! But it wasn't just go for a coffee and come back later

                                   

                                  I mean just let it work - hopefully after  a while it will complete 100%

                                  1 2 Previous Next