VMware Cloud Community
Attivoadmin
Contributor
Contributor

Unable to login to Vcenter

Hi

I have a vcenter server installed on a Windows server system. The vcenter server is not starting, it is showing an error message as

vSphere Client could not connect to X.X.X.X

An unknown connection error occurred. (The server could not interpret the client's request. (The remote server returned an error: (503) Server Unavailable.))

Request help at the earliest as this is one of the production vcente server.

Thank you.

Srikanthan

Reply
0 Kudos
24 Replies
Nawals
Expert
Expert

change it delay restart and restart web client service ( that was also stopped) after 1-2 minutes vpxd started it self.

it also take 5-10 minutes for initializing )

If resolve please mark helpful or correct.

NKS Please Mark Helpful/correct if my answer resolve your query.
Reply
0 Kudos
Attivoadmin
Contributor
Contributor

Hi Nawals,

Thank you for your quick reply, many of the services are not starting. The following are the services which are showing as stopped.

INFO:root:Service: VMwareComponentManager, Action: status

Service: VMwareComponentManager, Action: status

2020-03-17T07:35:20.144Z   State for service VMwareComponentManager: START_PENDING

INFO:root:Service: vmware-license, Action: status

Service: vmware-license, Action: status

2020-03-17T07:35:20.148Z   State for service vmware-license: STOPPED

INFO:root:Service: VMWareNetworkCoredumpWebserver, Action: status

Service: VMWareNetworkCoredumpWebserver, Action: status

2020-03-17T07:35:20.151Z   State for service VMWareNetworkCoredumpWebserver: STOPPED

INFO:root:Service: vmwareServiceControlAgent, Action: status

Service: vmwareServiceControlAgent, Action: status

2020-03-17T07:35:20.156Z   State for service vmwareServiceControlAgent: STOPPED

INFO:root:Service: vmware-network-coredump, Action: status

Service: vmware-network-coredump, Action: status

2020-03-17T07:35:20.160Z   State for service vmware-network-coredump: STOPPED

INFO:root:Service: vapiEndpoint, Action: status

Service: vapiEndpoint, Action: status

2020-03-17T07:35:20.164Z   State for service vapiEndpoint: STOPPED

INFO:root:Service: vPostgres, Action: status

Service: vPostgres, Action: status

2020-03-17T07:35:20.168Z   State for service vPostgres: STOPPED

INFO:root:Service: invsvc, Action: status

Service: invsvc, Action: status

2020-03-17T07:35:20.171Z   State for service invsvc: STOPPED

INFO:root:Service: mbcs, Action: status

Service: mbcs, Action: status

2020-03-17T07:35:20.175Z   State for service mbcs: STOPPED

INFO:root:Service: vpxd, Action: status

Service: vpxd, Action: status

2020-03-17T07:35:20.179Z   State for service vpxd: STOPPED

INFO:root:Service: EsxAgentManager, Action: status

Service: EsxAgentManager, Action: status

2020-03-17T07:35:20.183Z   State for service EsxAgentManager: STOPPED

INFO:root:Service: vmware-autodeploy-waiter, Action: status

Service: vmware-autodeploy-waiter, Action: status

2020-03-17T07:35:20.187Z   State for service vmware-autodeploy-waiter: STOPPED

INFO:root:Service: vimPBSM, Action: status

Service: vimPBSM, Action: status

2020-03-17T07:35:20.190Z   State for service vimPBSM: STOPPED

INFO:root:Service: vdcs, Action: status

Service: vdcs, Action: status

2020-03-17T07:35:20.194Z   State for service vdcs: STOPPED

INFO:root:Service: vmsyslogcollector, Action: status

Service: vmsyslogcollector, Action: status

2020-03-17T07:35:20.197Z   State for service vmsyslogcollector: STOPPED

INFO:root:Service: vmware-vpx-workflow, Action: status

Service: vmware-vpx-workflow, Action: status

2020-03-17T07:35:20.201Z   State for service vmware-vpx-workflow: STOPPED

INFO:root:Service: VServiceManager, Action: status

Service: VServiceManager, Action: status

2020-03-17T07:35:20.205Z   State for service VServiceManager: STOPPED

INFO:root:Service: vspherewebclientsvc, Action: status

Service: vspherewebclientsvc, Action: status

2020-03-17T07:35:20.208Z   State for service vspherewebclientsvc: STOPPED

INFO:root:Service: vmware-perfcharts, Action: status

Service: vmware-perfcharts, Action: status

2020-03-17T07:35:20.212Z   State for service vmware-perfcharts: STOPPED

INFO:root:Service: vmwarevws, Action: status

Service: vmwarevws, Action: status

2020-03-17T07:35:20.216Z   State for service vmwarevws: STOPPED

INFO:root:StartPending:

VMwareComponentManager (VMware Component Manager)

StartPending:

VMwareComponentManager (VMware Component Manager)

INFO:root:Running:

VMWareAfdService (VMware Authentication Framework) VMWareCertificateService (VMware Certificate Service) VMWareDirectoryService (VMware Directory Service) VMwareIdentityMgmtService (VMware Identity Management Service) VMwareSTS (VMware Security Token Service) rhttpproxy (VMware HTTP Reverse Proxy) vmware-cis-config ()

Running:

VMWareAfdService (VMware Authentication Framework) VMWareCertificateService (VMware Certificate Service) VMWareDirectoryService (VMware Directory Service) VMwareIdentityMgmtService (VMware Identity Management Service) VMwareSTS (VMware Security Token Service) rhttpproxy (VMware HTTP Reverse Proxy) vmware-cis-config ()

INFO:root:Stopped:

EsxAgentManager (VMware ESX Agent Manager) VMWareNetworkCoredumpWebserver (VMware vSphere ESXi Dump Collector WebService) VServiceManager (VMware vService Manager) invsvc (VMware Inventory Service) mbcs (VMware Message Bus Configuration Service) vPostgres (VMware Postgres) vapiEndpoint (VMware vAPI Endpoint) vdcs (VMware Content Library Service) vimPBSM (VMware vSphere Profile-Driven Storage Service) vmsyslogcollector (VMware Syslog Collector) vmware-auto

deploy-waiter (VMware vSphere Auto Deploy Waiter) vmware-license (VMware License Service) vmware-network-coredump (VMware vSphere ESXi Dump Collector) vmware-perfcharts (VMware Performance Charts) vmware-vpx-workflow (VMware vCenter Workflow Manager) vmwareServiceControlAgent (VMware Service Control Agent) vmwarevws (VMware System and Hardware Health Manager) vpxd (VMware vCenter Server) vspherewebclientsvc ()

Stopped:

EsxAgentManager (VMware ESX Agent Manager) VMWareNetworkCoredumpWebserver (VMware vSphere ESXi Dump Collector WebService) VServiceManager (VMware vService Manager) invsvc (VMware Inventory Service) mbcs (VMware Message Bus Configuration Service) vPostgres (VMware Postgres) vapiEndpoint (VMware vAPI Endpoint) vdcs (VMware Content Library Service) vimPBSM (VMware vSphere Profile-Driven Storage Service) vmsyslogcollector (VMware Syslog Collector) vmware-auto

deploy-waiter (VMware vSphere Auto Deploy Waiter) vmware-license (VMware License Service) vmware-network-coredump (VMware vSphere ESXi Dump Collector) vmware-perfcharts (VMware Performance Charts) vmware-vpx-workflow (VMware vCenter Workflow Manager) vmwareServiceControlAgent (VMware Service Control Agent) vmwarevws (VMware System and Hardware Health Manager) vpxd (VMware vCenter Server) vspherewebclientsvc ()

Please help......

Reply
0 Kudos
Alex_Romeo
Leadership
Leadership

Hi,

what version of vcenter do you have?

[SOLVED] The remote server returned an error: (503) Server Unavailable - VMware Forum - Spiceworks

ARomeo

Blog: https://www.aleadmin.it/
Reply
0 Kudos
Nawals
Expert
Expert

Please stop all services command : service-control --stop --all

once all stop services please start: service-control --start --all

If services not come up please reboot the vCenter and check . Also, check the space is available on database server where have the VCDB if vCenter on windows platform. Have any changes on the vCenter server?

NKS Please Mark Helpful/correct if my answer resolve your query.
Reply
0 Kudos
Lalegre
Virtuoso
Virtuoso

Try to start all the services and it will show specifically the one that is failing. After that go into the logs /var/vmware/log and search for the service failing.

Paste the output here and lets see.

Reply
0 Kudos
Attivoadmin
Contributor
Contributor

Hi Lalegre,

Tried starting it is throwing error message as unable to start VMWareComponentManager service.

I have installed it on a Windows setup, so please let me know how to get the logs.

Reply
0 Kudos
Attivoadmin
Contributor
Contributor

2020-03-18T07:22:07.691Z   State for service VMwareComponentManager: START_PENDI

NG

2020-03-18T07:22:07.691Z   ERROR Starting service: VMwareComponentManager, Excep

tion: (1056, 'StartService', 'An instance of the service is already running.')

ERROR:root:Unable to start service VMwareComponentManager, Exception: {

    "resolution": null,

    "detail": [

        {

            "args": [

                "VMwareComponentManager"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'VMwareCompon

entManager'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

Unable to start service VMwareComponentManager, Exception: {

    "resolution": null,

    "detail": [

        {

            "args": [

                "VMwareComponentManager"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'VMwareCompon

entManager'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

Reply
0 Kudos
Lalegre
Virtuoso
Virtuoso

Yes sure go to the next path: C:\ProgramData\VMware\CIS\logs or C:\ProgramData\VMware

There you will need to find the Component Manger folder that states as vmware-cm or CM only.

Inside there order the files by last date and copy the output or the entire log as attached file.

Reply
0 Kudos
Attivoadmin
Contributor
Contributor

Thank you for helping me, please find the log files.

Reply
0 Kudos
Lalegre
Virtuoso
Virtuoso

No worries man i am here to help you.

Please try to copy the cm.log becasue the one that you provide me is the cm.log.1 that only has 3 lines and it shows no error.

If you can not find it just export the logs and i will review them.

Reply
0 Kudos
Attivoadmin
Contributor
Contributor

Please find the log file.

Reply
0 Kudos
Lalegre
Virtuoso
Virtuoso

Attivo please show the output of the file eam.properties located it %ProgramData%\VMware\vCenterServer\cfg\vmware-eam.

Also check if the vmware-eam service is started and if not try to start it. If it fails share the log of that service.

Last please check if you have space on your Windows disks.

Reply
0 Kudos
Attivoadmin
Contributor
Contributor

Please find the eam file attached, tried starting the service, it is stuck for a long time and not proceeding further.

Reply
0 Kudos
Attivoadmin
Contributor
Contributor

Please find the eam log files, it never started.

Reply
0 Kudos
Lalegre
Virtuoso
Virtuoso

Try adding this line to eam.properties and try to start the vmware-eam service again (Please verify if that path is the correct regarding your installation)

eam.web.root=C:/Program Files/VMware/vCenter Server/eam/web

That line must be under this line:

tcserver.tmp.dir

Reply
0 Kudos
Attivoadmin
Contributor
Contributor

Made the changes to the eam properties file and tried starting, after a long time, this is the message it is showing up.

ERROR:root:Unable to start service EsxAgentManager, Exception: {

    "resolution": null,

    "detail": [

        {

            "args": [

                "EsxAgentManager"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'EsxAgentMana

ger'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

Unable to start service EsxAgentManager, Exception: {

    "resolution": null,

    "detail": [

        {

            "args": [

                "EsxAgentManager"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'EsxAgentMana

ger'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

Reply
0 Kudos
Attivoadmin
Contributor
Contributor

please find the attached eam logs

Reply
0 Kudos
Lalegre
Virtuoso
Virtuoso

Could you please paste my the output of the eam.properties file? I am getting out of ideas.

Reply
0 Kudos
Attivoadmin
Contributor
Contributor

#########################################################################

# Copyright 2013-2014 VMware, Inc.  All rights reserved. VMware Confidential

#########################################################################

vc.proxy.host=localhost

vc.proxy.port=80

# Hostname or IP of the EAM server

# Fill only if EAM is not running on the same host as VC

eam.host=

# EAM service port used to configure the HTTP connector of the application server.

eam.int.http.port=15005

# Port and scheme configuration which is used by the ESX 6.x hosts to reach EAM Vib

# file server.

eam.ext.port=443

eam.ext.scheme=https

# Port and scheme configuration which is used by the ESX 5.x hosts to reach EAM Vib

# file server.

eam.ext.port.deprecated=80

eam.ext.scheme.deprecated=http

eam.support_linked_clone=true

eam.clear_db_on_startup=false

eam.debug_ref_count=false

eam.recent_event_size=20

# Enable/disable VUM integration

vum.integration=true

# Value is specified in minutes (set to 24h = 1440m)

eam.scan_for_unknown_agent_vms=1440

# The timeout to wait for hostd to restart on a host (set to 5m=300s)

eam.hostd_restart_timeout=300

# The following entries will be added verbatim to the advanced options

# of hosts on which EAM is enabled. All are optional.

Net.DVFilterBindIpAddress=169.254.0.1

Net.TrafficFilterIpAddress=

#The IP for the VSWIF NIC on the dvFilter switch (for ESX classic).

Net.DVFilterVswifIpAddress=169.254.0.2

# Resource bundle configuration

eam.resourcebundle.filename=eam-resourcebundle.jar

# VLSI embedded tcServer configuration

#

tcserver.tmp.dir=C:/ProgramData/VMware/vCenterServer/runtime/tmp/vmware/eam/tomcat

eam.web.root=C:/Program Files/VMware/vCenter Server/eam/web

# EAM SSL configuration

#

eam.keystore.type=VKS

eam.key.alias=vpxd-extension

eam.keystore.storename=vpxd-extension

# CM configuration

#

cm.url=http://localhost:18090/cm/sdk/?hostid=f416ae50-62e8-11e5-812b-005056a2166a

cm.wait.attempts=360

cm.wait.intervalSeconds=5

# SSO configuration

#

sso.wait.attempts=360

sso.wait.intervalSeconds=5

# VC SSL configuration

#

vc.truststore.type=VKS

vc.truststore.storename=TRUSTED_ROOTS

vc.tunnelSdkUri.template=https://##{VC_HOST_NAME}##:8089/sdk/vimService

vc.tunnelSdkUri=https://10.16.1.201:8089/sdk/vimService

drs.demandCapacityRatio=100

Reply
0 Kudos