ESXI 6.5.0 vcenter 503 Service Unavailable error

[jasonbbcah39]

After rebooting our esxi hosts we get this browser error when we try to connect to the vsphere url

503 Service Unavailable (Failed to connect to endpoint: [class Vmacore::Http::NamedPipeServiceSpec:0x000000665c888a60] _serverNamespace = / action = Allow _pipeName =\\.\pipe\vmware-vpxd-webserver-pipe)

Attached is a log file.

Anyone have any ideas?

Cheers

Jason

[a_p_]

Welcome to the Community,

For how long did you wait after the reboot, before you tried to open the URL.

It usually takes a few minutes for the web server to initialize.

André

[admin]

vCenter 6.5 - duplicate key value violates unique constraint “pk_vpx_vm_virtual_device”

Regards,

Randhir

[msripada]

The logs which you uploaded shows the below errors

at com.vmware.vim.dataservices.WrapperListenerImpl$Main_Vmon.start(WrapperListenerImpl.java:157)

at com.vmware.vim.dataservices.WrapperListenerImpl.main(WrapperListenerImpl.java:141)

Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:123)

at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)

at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:124)

at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)

at org.postgresql.jdbc3.Jdbc3Connection.<init>(Jdbc3Connection.java:24)

at org.postgresql.Driver.makeConnection(Driver.java:386)

at org.postgresql.Driver.connect(Driver.java:260)

at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)

at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)

at org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1556)

at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1545)

... 96 more

Caused by: java.net.ConnectException: Connection refused: connect

at java.net.DualStackPlainSocketImpl.connect0(Native Method)

at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)

at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)

at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)

at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)

at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)

at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

Can you run the command service-control --status --all and check which service is down and share the output.

Can you check if rhttpproxy service is started, if not try to start it and check if that is working

thanks,

MS

[jasonbbcah39]

Hi,

Thanks for looking at the logs.

Here is the service status output.

Running:

EsxAgentManager

VMWareAfdService

VMWareCertificateService

VMWareDirectoryService

VMwareComponentManager

VMwareDNSService

VMwareIdentityMgmtService

VMwareSTS rhttpproxy

vapiEndpoint

vmon

vmonapi

vmware-cis-config

vmware-license

vmware-psc-client

vmwareServiceControlAgent

vsphere-ui vspherewebclientsvc

Stopped:

VMWareCAMService

VServiceManager

content-library

mbcs

vPostgres

vimPBSM

vmsyslogcollector

vmware-autodeploy-waiter

vmware-imagebuilder

vmware-network-coredump

vmware-perfcharts

vpxd

vpxd-svcs

vsan-health

Regards

Jason

[jasonbbcah39]

Thank-you Andre

We waited around 20 minutes. Still no joy I'm afraid.

Regards

Jason

[jasonbbcah39]

Hi Randhir

Interesting, I will have a look.

Regards

Jason

[Finikiez]

Hi!

503 - is a general error. There are could be a lot of things behind it.

From service-control output I see that 'vpxd' service is stopped. It's main vcenter service.

Can you attach the latest vpxd.log here so I could check what's wrong with it.

[msripada]

postgres is stopped... i assume there is something wrong with the database or corruption in that... can you check the latest postgresql log for more information.Without postgres, vpxd do not start...

Thanks,

MS

[rcaljouw]

Hi,

Last time we had this issue, our time server was not available.

Changed NTP services on both vCenter (vsc and psc) and ESX Horizon hosts to working NTP time server solved issue.

Cheers

Ronald

[NRay]

A new comprehensive article has been created on kb.vmware.com for this issue.

Please see the KB here: esxi 6.5 503 service unavailable (failed to connect to endpoint