Rambo21
Contributor
Contributor

503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http20NamedPipeServiceSpecE:0x000055ed2fdb3820] _serverNamespace = / action = Allow _pipeName =/var/run/vmware/vpxd-webserver-pipe)

503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http20NamedPipeServiceSpecE:0x000055ed2fdb3820] _serverNamespace = / action = Allow _pipeName =/var/run/vmware/vpxd-webserver-pipe)

VMware vCenter Server Appliance 6.7.0.10000

NSX manager 6.4.0

13 Replies
Rambo21
Contributor
Contributor

root@172 [ ~ ]# service-control --status --all

Running:

applmgmt lwsmd vmafdd vmcad vmdird vmdnsd vmonapi vmware-analytics vmware-cis-license vmware-eam vmware-netdumper vmware-postgres-archiver vmware-rbd-watchdog vmware-rhttpproxy vmware-statsmonitor vmware-sts-idmd vmware-stsd vmware-vmon vmware-vpostgres vsphere-client vsphere-ui

Stopped:

pschealth vmcam vmware-cm vmware-content-library vmware-imagebuilder vmware-mbcs vmware-perfcharts vmware-pod vmware-sca vmware-sps vmware-updatemgr vmware-vapi-endpoint vmware-vcha vmware-vpxd vmware-vpxd-svcs vmware-vsan-health vmware-vsm vsan-dps

service-control --status --start  

Service-control failed. Error: Failed to start services in profile ALL. RC=1, stderr=Failed to start sca, vapi-endpoint, cm services. Error: Operation timed out

0 Kudos
daphnissov
Immortal
Immortal

Did you install your vCenter using a FQDN and have DNS records (forward and reverse) present in your DNS server?

msripada
Virtuoso
Virtuoso

is it vcenter with external psc or embedded psc?

can you check the vmware-cm log - var/log/vmware/cm

Thanks,

MS

0 Kudos
meskirian
Contributor
Contributor

it looks like i got the same problem:

503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http20NamedPipeServiceSpecE:0x00007fbde8005fc0] _serverNamespace = / action = Allow _pipeName =/var/run/vmware/vpxd-webserver-pipe)

StartPending:

vmware-vsan-health

Running:

applmgmt lwsmd vmafdd vmcad vmdird vmdnsd vmonapi vmware-psc-client vmware-rhttpproxy vmware-statsmonitor vmware-sts-idmd vmware-stsd vmware-vcha vmware-vmon vmware-vpostgres vsphere-client vsphere-ui

Stopped:

pschealth vmcam vmware-cis-license vmware-cm vmware-content-library vmware-eam vmware-imagebuilder vmware-mbcs vmware-netdumper vmware-perfcharts vmware-rbd-watchdog vmware-sca vmware-sps vmware-updatemgr vmware-vapi-endpoint vmware-vpxd vmware-vpxd-svcs vmware-vsm

Error executing start on service vpxd. Details {

    "resolution": null,

    "detail": [

        {

            "args": [

                "vpxd"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'vpxd'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

Service-control failed. Error {

    "resolution": null,

    "detail": [

        {

            "args": [

                "vpxd"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'vpxd'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

Could any one help me?

vcenter is embedded

P.S.

i tried solution from the post VCSA 6.5 Web GUI Error

and it didn't help.

0 Kudos
Mokshir
Enthusiast
Enthusiast

Running into the same issue.  vCenter was running fine until a 2 weeks ago. The it died. I found out it ran out space to expan the swap file so I reserved all the memory for the VM. VM came up and ran into this error

503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http20NamedPipeServiceSpecE:0x00005650c59d5e50] _serverNamespace = / action = Allow _pipeName =/var/run/vmware/vpxd-webserver-pipe)

In the VPXD logs I see the following:

76 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key log.level, pre commit

2018-05-24T18:50:27.852Z info vpxd[7F3B33476800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key log.level, pre commit

2018-05-24T18:50:27.852Z info vpxd[7F3B33476800] [Originator@6876 sub=MoOptionMgr] No change to log.level

2018-05-24T18:50:27.858Z info vpxd[7F3B33476800] [Originator@6876 sub=Main] [VpxdMain] Setting OpenSSL verify locations CAFile= CAPath=/etc/ssl/certs

2018-05-24T18:50:27.858Z info vpxd[7F3B33476800] [Originator@6876 sub=Default] Creating SSL Contexts

2018-05-24T18:50:27.886Z info vpxd[7F3B33476800] [Originator@6876 sub=WorkQueue.vpxd.stats.persister.queue] Created: WorkQueue.vpxd.stats.persister.queue, type = serial, priority = 16 , itemWeight = 1

2018-05-24T18:50:27.891Z info vpxd[7F3B33476800] [Originator@6876 sub=DbParallelLoader] Num rows: 7, num CPUs: 4, threads: 1, step: 7

2018-05-24T18:50:27.941Z error vpxd[7F3B33476800] [Originator@6876 sub=DbBulkLoader] [VpxdDbBulkLoader::Load] Failed to load tableDef 41 from database: Column DVS_ID does not exist in table VPX_DVS_NRP

Tried restarting the service. verified FQDN resolution no luck. Seems like a DVS issue to which this vCenter VM is connected to but I have not clue how to fix this.

Help.

0 Kudos
grep_boy
Contributor
Contributor

I had the same issue, which finally turned out to be a certificate problem.

I fixed by logging into VCSA as root, and running the certificate manager: /usr/lib/vmware-vmca/bin/certificate-manager

Good KB article at https://kb.vmware.com/kb/2097936

I used option 3 to regenerate all of the SSL certificates which got me up and running.

josearpa
Contributor
Contributor

Hi.

I found that the process update failed because root password expired.  I changed root password and start update process from ssh.

Good luck.

0 Kudos
msripada
Virtuoso
Virtuoso

For all our community members,

Error 503 is Service Unavailable when trying to connect to vCenter server service or it can be webclient as well.

This error is very generic in nature however possible troubleshootings include if the services are started or failed. If failed, look into respective logs.

In your case, the update failed so expected the service to be failed.

Other cases, service might be crashed or certificates expired or partition full issues.. It can vary from vCenter to vCenter and environment to environment.

Hope this helps for whoever trying to troubleshoot.

Thanks,

MS

ammadkazmi1
Contributor
Contributor

Thank you...... It has resolve my issue

Ammad Kazmi

0 Kudos
apatten
Contributor
Contributor

I'll add that I had this same issue and it turned out to be a issue with name resolution.

From the /var/log/vmware/cm/cm.log file:

2019-05-22T13:06:33.537Z [main [] WARN  com.vmware.cis.services.cm.service.util.LsUtils ()] Call to lookup service failed; uri:https://<my vcsa's hostname>/lookupservice/sdk [com.vmware.vim.vmomi.client.exception.ConnectionException: java.net.UnknownHostException: <my vcsa's hostname>]

I added an entry in the /etc/hosts file and that resolved the issue after restarting the appliance.

0 Kudos
blackcopy
Contributor
Contributor

In my case, the /storage/seat partition was filled over 95%-threshold. Increasing space helped.

I found it out by digging through the vpxd.log

sreeve3939
Enthusiast
Enthusiast

Apatten: what entries did you put in your /etc/host file?

Here's mine right now - still having the problem as the OP.

pastedImage_0.png

0 Kudos
Jack_Moon
VMware Employee
VMware Employee

Thanks for sharing your experiences and i solve this issue to create a new certificate with running /usr/lib/vmware-vmca/bin/certificate-manager as you guided. Thank! grep_boy

0 Kudos