10 Replies Latest reply on Oct 3, 2019 7:10 AM by VirtualredSE

    Can't access VCSA from web anymore, "connection refused". Strange errors when running "service-control --status --all"

    Teemuzza Lurker

      Hello, this is the first forum post i have ever made in my life, but i really cant figure this one on my own. I would really appreciate the help, if someone knows how to!

      I recently moved my 6.7 ESXi hosts and 6.7 vCenter server to a distributed switch. Everything seemed to go fine, I can still access my 3 hosts form web and i can ping everything (including VCSA). However, i can't access VCSA from web anymore, with Chrome i just get a message saying 10.69.51.100 refused to connect. Can't access it with any other browser either. I've tried rebooting my vCenter multiple times, didn't help at all. I also tried checking if all the services are up and running. However, then i get these strange errors i couldn't find pretty much any info about online:

       

      Command> service-control --status --all

      2018-06-16T11:58:41.796Z  Error: Service name "applmgmt" is invalid.

      2018-06-16T11:58:41.816Z  Error: Service name "content-library" is invalid.

      2018-06-16T11:58:41.837Z  Error: Service name "vcha" is invalid.

      2018-06-16T11:58:41.857Z  Error: Service name "pschealth" is invalid.

      2018-06-16T11:58:41.878Z  Error: Service name "sps" is invalid.

      2018-06-16T11:58:41.898Z  Error: Service name "sca" is invalid.

      2018-06-16T11:58:41.936Z  Error: Service name "vmcam" is invalid.

      2018-06-16T11:58:41.956Z  Error: Service name "vmware-vpostgres" is invalid.

      2018-06-16T11:58:41.976Z  Error: Service name "vsphere-client" is invalid.

      2018-06-16T11:58:41.997Z  Error: Service name "vapi-endpoint" is invalid.

      2018-06-16T11:58:42.017Z  Error: Service name "vmware-postgres-archiver" is invalid.

      2018-06-16T11:58:42.037Z  Error: Service name "vsm" is invalid.

      2018-06-16T11:58:42.057Z  Error: Service name "updatemgr" is invalid.

      2018-06-16T11:58:42.078Z  Error: Service name "vmonapi" is invalid.

      2018-06-16T11:58:42.098Z  Error: Service name "vsan-health" is invalid.

      2018-06-16T11:58:42.119Z  Error: Service name "rbd" is invalid.

      2018-06-16T11:58:42.156Z  Error: Service name "vpxd-svcs" is invalid.

      2018-06-16T11:58:42.176Z  Error: Service name "statsmonitor" is invalid.

      2018-06-16T11:58:42.264Z  Error: Service name "imagebuilder" is invalid.

      2018-06-16T11:58:42.284Z  Error: Service name "vsan-dps" is invalid.

      2018-06-16T11:58:42.304Z  Error: Service name "cm" is invalid.

      2018-06-16T11:58:42.342Z  Error: Service name "mbcs" is invalid.

      2018-06-16T11:58:42.362Z  Error: Service name "vpxd" is invalid.

      2018-06-16T11:58:42.382Z  Error: Service name "netdumper" is invalid.

      2018-06-16T11:58:42.403Z  Error: Service name "rhttpproxy" is invalid.

      2018-06-16T11:58:42.423Z  Error: Service name "vsphere-ui" is invalid.

      2018-06-16T11:58:42.444Z  Error: Service name "eam" is invalid.

      2018-06-16T11:58:42.464Z  Error: Service name "perfcharts" is invalid.

      2018-06-16T11:58:42.519Z  Error: Service name "analytics" is invalid.

      2018-06-16T11:58:42.539Z  Error: Service name "cis-license" is invalid.

      Running:

      lwsmd vmafdd vmcad vmdird vmdnsd vmware-pod vmware-sts-idmd vmware-stsd vmware-vmon

       

      Could these be a part of the problem? If i try to start all the services, it only starts the ones already running again, doesen't even try to start those others with the invalid name.

       

      Command> service-control --start --all

      Operation not cancellable. Please wait for it to finish...

      Performing start operation on service lwsmd...

      Successfully started service lwsmd

      Performing start operation on service vmafdd...

      Successfully started service vmafdd

      Performing start operation on service vmdird...

      Successfully started service vmdird

      Performing start operation on service vmcad...

      Successfully started service vmcad

      Performing start operation on service vmware-sts-idmd...

      Successfully started service vmware-sts-idmd

      Performing start operation on service vmware-stsd...

      Successfully started service vmware-stsd

      Performing start operation on service vmdnsd...

      Successfully started service vmdnsd

      Performing start operation on profile: ALL...

      Successfully started profile: ALL.

      Performing start operation on service vmware-pod...

      Successfully started service vmware-pod

       

      I haven't made any actual changes to vCenter configuration, other than moving it to a dswitch. Everything worked just fine before. Oh, and i'm just on a home lab, trying to work on my practical part of my thesis work. Does anyone know what's happening here? I'm happy to provide you with more information if needed/possible.

        • 1. Re: Can't access VCSA from web anymore, "connection refused". Strange errors when running "service-control --status --all"
          jakesmorrison Lurker

          I am having the exact same issues. I have deleted and then recreated multiple vCenters. Every time I shutdown my ESXI server the vCenter gets corrupted somehow. I thought I figured this out by forcing the vCenter to power up on startup, but I came in this morning and can no longer access vCenter.

           

          This is definitely the issues; none of the services are starting.

           

          root@192 [ ~ ]# service-control --status

          2018-06-26T08:05:05.850Z  Error: Service name "vsphere-client" is invalid.

          2018-06-26T08:05:05.870Z  Error: Service name "imagebuilder" is invalid.

          2018-06-26T08:05:05.890Z  Error: Service name "perfcharts" is invalid.

          2018-06-26T08:05:05.909Z  Error: Service name "vmonapi" is invalid.

          2018-06-26T08:05:05.928Z  Error: Service name "vcha" is invalid.

          2018-06-26T08:05:05.948Z  Error: Service name "rbd" is invalid.

          2018-06-26T08:05:05.967Z  Error: Service name "vmcam" is invalid.

          2018-06-26T08:05:05.986Z  Error: Service name "analytics" is invalid.

          2018-06-26T08:05:06.006Z  Error: Service name "applmgmt" is invalid.

          2018-06-26T08:05:06.025Z  Error: Service name "vmware-postgres-archiver" is invalid.

          2018-06-26T08:05:06.044Z  Error: Service name "vmware-vpostgres" is invalid.

          2018-06-26T08:05:06.106Z  Error: Service name "rhttpproxy" is invalid.

          2018-06-26T08:05:06.168Z  Error: Service name "vapi-endpoint" is invalid.

          2018-06-26T08:05:06.207Z  Error: Service name "cis-license" is invalid.

          2018-06-26T08:05:06.246Z  Error: Service name "sps" is invalid.

          2018-06-26T08:05:06.266Z  Error: Service name "updatemgr" is invalid.

          2018-06-26T08:05:06.285Z  Error: Service name "statsmonitor" is invalid.

          2018-06-26T08:05:06.333Z  Error: Service name "vsm" is invalid.

          2018-06-26T08:05:06.353Z  Error: Service name "cm" is invalid.

          2018-06-26T08:05:06.372Z  Error: Service name "sca" is invalid.

          2018-06-26T08:05:06.409Z  Error: Service name "eam" is invalid.

          2018-06-26T08:05:06.428Z  Error: Service name "content-library" is invalid.

          2018-06-26T08:05:06.468Z  Error: Service name "vsan-health" is invalid.

          2018-06-26T08:05:06.487Z  Error: Service name "vsphere-ui" is invalid.

          2018-06-26T08:05:06.506Z  Error: Service name "vpxd-svcs" is invalid.

          2018-06-26T08:05:06.526Z  Error: Service name "vsan-dps" is invalid.

          2018-06-26T08:05:06.545Z  Error: Service name "mbcs" is invalid.

          2018-06-26T08:05:06.564Z  Error: Service name "pschealth" is invalid.

          2018-06-26T08:05:06.584Z  Error: Service name "netdumper" is invalid.

          2018-06-26T08:05:06.603Z  Error: Service name "vpxd" is invalid.

          Running:

          lwsmd vmafdd vmcad vmdird vmdnsd vmware-pod vmware-sts-idmd vmware-stsd vmware-vmon

           

          I cannot manually start any process. I get an error.

           

          service-control --start vsphere-client

          Operation not cancellable. Please wait for it to finish...

          Performing start operation on service vsphere-client...

          Successfully started service vmware-vmon

          2018-06-26T08:17:19.228Z  Error: Invalid input provided to get startType ofservice vsphere-client.

          Error executing start on service vsphere-client. Details Error: startType of svc: vsphere-client rc : 6 stdout:  stderr: Service Get State request failed. Error: Invalid input data

          Service-control failed. Error: Error: startType of svc: vsphere-client rc : 6 stdout:  stderr: Service Get State request failed. Error: Invalid input data

          • 2. Re: Can't access VCSA from web anymore, "connection refused". Strange errors when running "service-control --status --all"
            Kelkin Novice

            Did you ever find a solution to this? I'm having the same exact error. I just installed the VCSA appliance yesterday and suddenly couldn't access it anymore. I wiped it out, reinstalled it this morning, didn't even configure anything once it was up yet, and tonight the same exact thing.

             

            Command> shell

            Shell access is granted to root

            root@VCenter01 [ ~ ]# service-control --start --all

            Operation not cancellable. Please wait for it to finish...

            Performing start operation on service lwsmd...

            Successfully started service lwsmd

            Performing start operation on service vmafdd...

            Successfully started service vmafdd

            Performing start operation on service vmdird...

            Successfully started service vmdird

            Performing start operation on service vmcad...

            Successfully started service vmcad

            Performing start operation on service vmware-sts-idmd...

            Successfully started service vmware-sts-idmd

            Performing start operation on service vmware-stsd...

            Successfully started service vmware-stsd

            Performing start operation on service vmdnsd...

            Successfully started service vmdnsd

            Performing start operation on profile: ALL...

            Successfully started profile: ALL.

            Performing start operation on service vmware-pod...

            Successfully started service vmware-pod

            root@VCenter01 [ ~ ]# service-control --status --all

            2018-10-08T07:18:25.293Z  Error: Service name "eam" is invalid.

            2018-10-08T07:18:25.306Z  Error: Service name "sps" is invalid.

            2018-10-08T07:18:25.320Z  Error: Service name "vmware-postgres-archiver" is invalid.

            2018-10-08T07:18:25.332Z  Error: Service name "vpxd" is invalid.

            2018-10-08T07:18:25.345Z  Error: Service name "cm" is invalid.

            2018-10-08T07:18:25.358Z  Error: Service name "content-library" is invalid.

            2018-10-08T07:18:25.370Z  Error: Service name "vsphere-ui" is invalid.

            2018-10-08T07:18:25.383Z  Error: Service name "vcha" is invalid.

            2018-10-08T07:18:25.396Z  Error: Service name "vmware-vpostgres" is invalid.

            2018-10-08T07:18:25.409Z  Error: Service name "vmonapi" is invalid.

            2018-10-08T07:18:25.422Z  Error: Service name "vsan-health" is invalid.

            2018-10-08T07:18:25.434Z  Error: Service name "pschealth" is invalid.

            2018-10-08T07:18:25.447Z  Error: Service name "rhttpproxy" is invalid.

            2018-10-08T07:18:25.460Z  Error: Service name "applmgmt" is invalid.

            2018-10-08T07:18:25.473Z  Error: Service name "updatemgr" is invalid.

            2018-10-08T07:18:25.486Z  Error: Service name "perfcharts" is invalid.

            2018-10-08T07:18:25.512Z  Error: Service name "imagebuilder" is invalid.

            2018-10-08T07:18:25.525Z  Error: Service name "vapi-endpoint" is invalid.

            2018-10-08T07:18:25.560Z  Error: Service name "netdumper" is invalid.

            2018-10-08T07:18:25.573Z  Error: Service name "vmcam" is invalid.

            2018-10-08T07:18:25.619Z  Error: Service name "rbd" is invalid.

            2018-10-08T07:18:25.632Z  Error: Service name "vsphere-client" is invalid.

            2018-10-08T07:18:25.655Z  Error: Service name "statsmonitor" is invalid.

            2018-10-08T07:18:25.680Z  Error: Service name "mbcs" is invalid.

            2018-10-08T07:18:25.704Z  Error: Service name "cis-license" is invalid.

            2018-10-08T07:18:25.716Z  Error: Service name "vsan-dps" is invalid.

            2018-10-08T07:18:25.729Z  Error: Service name "sca" is invalid.

            2018-10-08T07:18:25.742Z  Error: Service name "vpxd-svcs" is invalid.

            2018-10-08T07:18:25.755Z  Error: Service name "vsm" is invalid.

            2018-10-08T07:18:25.768Z  Error: Service name "analytics" is invalid.

            Running:

            lwsmd vmafdd vmcad vmdird vmdnsd vmware-pod vmware-sts-idmd vmware-stsd vmware-vmon

            root@VCenter01 [ ~ ]# find /etc/systemd/system/ -lname '/dev/null' -exec rm {} ;

            find: missing argument to `-exec'

            root@VCenter01 [ ~ ]# shell

            bash: shell: command not found

            root@VCenter01 [ ~ ]# find /etc/systemd/system/ -lname '/dev/null' -exec rm {} ;

            find: missing argument to `-exec'

            root@VCenter01 [ ~ ]#

            root@VCenter01 [ ~ ]# systemctl daemon-reload

            root@VCenter01 [ ~ ]#

            root@VCenter01 [ ~ ]# find /etc/systemd/system/ -lname '/dev/null' -exec rm {} ;

            find: missing argument to `-exec'

            root@VCenter01 [ ~ ]# find /etc/systemd/system/ -lname '/dev/null' -exec rm {} ;

            find: missing argument to `-exec'

            root@VCenter01 [ ~ ]#

            root@VCenter01 [ ~ ]# systemctl daemon-reload

            root@VCenter01 [ ~ ]#

            root@VCenter01 [ ~ ]# service-control --start --all

            Operation not cancellable. Please wait for it to finish...

            Performing start operation on service lwsmd...

            Successfully started service lwsmd

            Performing start operation on service vmafdd...

            Successfully started service vmafdd

            Performing start operation on service vmdird...

            Successfully started service vmdird

            Performing start operation on service vmcad...

            Successfully started service vmcad

            Performing start operation on service vmware-sts-idmd...

            Successfully started service vmware-sts-idmd

            Performing start operation on service vmware-stsd...

            Successfully started service vmware-stsd

            Performing start operation on service vmdnsd...

            Successfully started service vmdnsd

            Performing start operation on profile: ALL...

            Successfully started profile: ALL.

            Performing start operation on service vmware-pod...

            Successfully started service vmware-pod

            root@VCenter01 [ ~ ]# service-control --start vsphere-client

            Operation not cancellable. Please wait for it to finish...

            Performing start operation on service vsphere-client...

            2018-10-08T07:24:38.502Z  Error: Invalid input provided to get startType ofservice vsphere-client.

            Error executing start on service vsphere-client. Details Error: startType of svc: vsphere-client rc : 6 stdout:  stderr: Service Get State request failed. Error: Invalid input data

             

            Service-control failed. Error: Error: startType of svc: vsphere-client rc : 6 stdout:  stderr: Service Get State request failed. Err

            • 3. Re: Can't access VCSA from web anymore, "connection refused". Strange errors when running "service-control --status --all"
              mdahili Lurker

              I have a setup simulating 2 DC based on 1 x VCSA and 1 x PSC (external) in each DC.

              I started by upgrading the PSC then the VCSA from 6.5 to 6.7. The ESXI servers were still running 6.5. No issue. I was able to access both the VCSA.

              I powered them off and started upgrading the ESXI servers from 6.5 to 6.7. No issue after reboot.

              After restarting the PSC and VCSA in each DC, only VCSA in DC2 is reachable. VCSA in DC1 has the same issue:

               

              root@csl-dc1-vcsa01 [ /bin ]# service-control --status --all

              2018-10-09T16:14:49.873Z  Error: Service name "vsphere-client" is invalid.

              2018-10-09T16:14:49.911Z  Error: Service name "vsan-dps" is invalid.

              2018-10-09T16:14:49.935Z  Error: Service name "vmware-postgres-archiver" is invalid.

              2018-10-09T16:14:49.978Z  Error: Service name "vpxd" is invalid.

              2018-10-09T16:14:50.001Z  Error: Service name "vcha" is invalid.

              2018-10-09T16:14:50.024Z  Error: Service name "vsm" is invalid.

              2018-10-09T16:14:50.068Z  Error: Service name "vmonapi" is invalid.

              2018-10-09T16:14:50.091Z  Error: Service name "vmcam" is invalid.

              2018-10-09T16:14:50.114Z  Error: Service name "imagebuilder" is invalid.

              2018-10-09T16:14:50.137Z  Error: Service name "rbd" is invalid.

              2018-10-09T16:14:50.161Z  Error: Service name "vmware-vpostgres" is invalid.

              2018-10-09T16:14:50.184Z  Error: Service name "rhttpproxy" is invalid.

              2018-10-09T16:14:50.228Z  Error: Service name "sps" is invalid.

              2018-10-09T16:14:50.251Z  Error: Service name "statsmonitor" is invalid.

              2018-10-09T16:14:50.274Z  Error: Service name "vsan-health" is invalid.

              2018-10-09T16:14:50.297Z  Error: Service name "vapi-endpoint" is invalid.

              2018-10-09T16:14:50.320Z  Error: Service name "netdumper" is invalid.

              2018-10-09T16:14:50.344Z  Error: Service name "vpxd-svcs" is invalid.

              2018-10-09T16:14:50.367Z  Error: Service name "cm" is invalid.

              2018-10-09T16:14:50.411Z  Error: Service name "applmgmt" is invalid.

              2018-10-09T16:14:50.435Z  Error: Service name "eam" is invalid.

              2018-10-09T16:14:50.458Z  Error: Service name "content-library" is invalid.

              2018-10-09T16:14:50.481Z  Error: Service name "perfcharts" is invalid.

              2018-10-09T16:14:50.505Z  Error: Service name "updatemgr" is invalid.

              2018-10-09T16:14:50.529Z  Error: Service name "vsphere-ui" is invalid.

              2018-10-09T16:14:50.552Z  Error: Service name "analytics" is invalid.

              2018-10-09T16:14:50.575Z  Error: Service name "mbcs" is invalid.

              2018-10-09T16:14:50.598Z  Error: Service name "sca" is invalid.

              Running:

              lwsmd vmafdd vmware-pod vmware-vmon

               

              root@csl-dc1-vcsa01 [ /bin ]# service-control --start vsphere-client

              Operation not cancellable. Please wait for it to finish...

              Performing start operation on service vsphere-client...

              2018-10-09T16:14:58.942Z  Error: Invalid input provided to get startType ofservice vsphere-client.

              Error executing start on service vsphere-client. Details Error: startType of svc: vsphere-client rc : 6 stdout:  stderr: Service Get State request failed. Error: Invalid input data

              Service-control failed. Error: Error: startType of svc: vsphere-client rc : 6 stdout:  stderr: Service Get State request failed. Error: Invalid input data

               

              Would someone have a solution?

              • 5. Re: Can't access VCSA from web anymore, "connection refused". Strange errors when running "service-control --status --all"
                ACBelieve Lurker

                There is no way to fix it unless reinstall VCSA.

                Next time, do not try to do anything while those services starting.

                Be sure always check your monitor.

                monitor.PNG

                It spend totally 16 min to start all services.

                • 6. Re: Can't access VCSA from web anymore, "connection refused". Strange errors when running "service-control --status --all"
                  msripada Expert
                  vExpert

                  I found this issue and a workaround for now. I seen an issue where due to storage or time sync issues on the VCSA or on the ESXi host where the vCSA is residing leading to the issue.

                   

                  Workaround you can try :

                   

                  Snapshot vCenter appliance and ensure backup is there before attempting the below steps

                   

                  cd /storage/vmware-vmon/.svcStats

                  Move the json files to another temporary location

                  Stop and start all services should work fine

                   

                  Ensure that the time sync and storage issues are fixed prior to performing this else it would reappear again.

                   

                  Thanks,

                  MS

                  1 person found this helpful