VMware Cloud Community
ChrisFD2
VMware Employee
VMware Employee
Jump to solution

ESX Agent Manager - vmware-eam will not start

First I noticed /etc/vmware-eam/eam.properties was empty, so I poulated it with the correct entries using another working vCSA (changed URLs and CM configuration URL with the correct hostid, but the service still does not start.

I see the following output if I try and start EAM via shell.

I have a VMware ticket open but they are being really slow at getting back to me.

Is it possible to remove the extension and add it again? I suspect that due to the disk running out of space a few weeks ago some files may have become corrupt.

service-control --start vmware-eam

INFO:root:Service: vmware-eam, Action: start

Service: vmware-eam, Action: start

2017-12-20T10:38:45.407Z   Running command: ['/sbin/chkconfig', u'vmware-eam']

2017-12-20T10:38:45.442Z   Done running command

2017-12-20T10:38:45.442Z   Running command: ['/sbin/service', u'vmware-eam', 'status']

2017-12-20T10:38:45.489Z   Done running command

2017-12-20T10:38:45.489Z   Running command: ['/sbin/chkconfig', '--force', u'vmware-eam', 'on']

2017-12-20T10:38:45.524Z   Done running command

2017-12-20T10:38:45.524Z   Running command: ['/sbin/service', u'vmware-eam', 'start']

2017-12-20T10:38:49.191Z   Done running command

2017-12-20T10:38:49.192Z   Invoked command: ['/sbin/service', u'vmware-eam', 'start']

2017-12-20T10:38:49.192Z   RC = 1

Stdout = Starting VMware ESX Agent Manager...

Waiting for VMware ESX Agent Manager......

WARNING: VMware ESX Agent Manager may have failed to start.

Last login: Wed Dec 20 10:31:58 UTC 2017 on pts/1

Stderr =

2017-12-20T10:38:49.192Z   {

    "resolution": null,

    "detail": [

        {

            "args": [

                "Command: ['/sbin/service', u'vmware-eam', 'start']\nStderr: "

            ],

            "id": "install.ciscommon.command.errinvoke",

            "localized": "An error occurred while invoking external command : 'Command: ['/sbin/service', u'vmware-eam', 'start']\nStderr: '",

            "translatable": "An error occurred while invoking external command : '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

ERROR:root:Unable to start service vmware-eam, Exception: {

    "resolution": null,

    "detail": [

        {

            "args": [

                "vmware-eam"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'vmware-eam'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

Unable to start service vmware-eam, Exception: {

    "resolution": null,

    "detail": [

        {

            "args": [

                "vmware-eam"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'vmware-eam'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

This is the content of /storage/log/vmware/eam/wrapper.log when I try and start it.

I have looked at the eam-server-beans.xml file and it is identical to a server that is working.

STATUS | wrapper  | 2017/12/20 11:26:30 | --> Wrapper Started as Daemon

STATUS | wrapper  | 2017/12/20 11:26:30 | Java Service Wrapper Professional Edition 64-bit 3.5.24

STATUS | wrapper  | 2017/12/20 11:26:30 |   Copyright (C) 1999-2014 Tanuki Software, Ltd. All Rights Reserved.

STATUS | wrapper  | 2017/12/20 11:26:30 |     http://wrapper.tanukisoftware.com

STATUS | wrapper  | 2017/12/20 11:26:30 |   Licensed to VMware Global, Inc. for VMware vCenter ESX Agent Manager

STATUS | wrapper  | 2017/12/20 11:26:30 |

INFO   | wrapper  | 2017/12/20 11:26:30 | Event Command 'wrapper_start': Command line[0] : /usr/sbin/cloudvm-ram-size

INFO   | wrapper  | 2017/12/20 11:26:30 | Event Command 'wrapper_start': Command line[1] : -J

INFO   | wrapper  | 2017/12/20 11:26:30 | Event Command 'wrapper_start': Command line[2] : vmware-eam

INFO   | wrapper  | 2017/12/20 11:26:30 | Event Command 'wrapper_start': Command line[3] : -O

INFO   | wrapper  | 2017/12/20 11:26:30 | Event Command 'wrapper_start': Command line[4] : /etc/vmware/java/vmware-eam_jvm.conf

INFO   | wrapper  | 2017/12/20 11:26:30 | Event Command 'wrapper_start': Command launched (pid: 12435), blocking for up to 3600 seconds...

INFO   | wrapper  | 2017/12/20 11:26:30 | Event Command 'wrapper_start': Command completed with exit code: 0

INFO   | wrapper  | 2017/12/20 11:26:30 | Event Command 'wrapper_start': Command completed with exit code: 0  Continuing.

INFO   | wrapper  | 2017/12/20 11:26:30 | Event Command 'jvm_prelaunch': Command line[0] : /opt/vmware/bin/python

INFO   | wrapper  | 2017/12/20 11:26:30 | Event Command 'jvm_prelaunch': Command line[1] : ../wrapper/bin/update_pnid.py

INFO   | wrapper  | 2017/12/20 11:26:30 | Event Command 'jvm_prelaunch': Command line[2] :

INFO   | wrapper  | 2017/12/20 11:26:30 | Event Command 'jvm_prelaunch': Command line[3] :

INFO   | wrapper  | 2017/12/20 11:26:30 | Event Command 'jvm_prelaunch': Command line[4] :

INFO   | wrapper  | 2017/12/20 11:26:30 | Event Command 'jvm_prelaunch': Command launched (pid: 12442), blocking for up to 15 seconds...

INFO   | wrapper  | 2017/12/20 11:26:30 | Event Command 'jvm_prelaunch': Command completed with exit code: 0

INFO   | wrapper  | 2017/12/20 11:26:30 | Event Command 'jvm_prelaunch': Command completed with exit code: 0  Continuing.

STATUS | wrapper  | 2017/12/20 11:26:30 | Launching a JVM...

INFO   | jvm 1    | 2017/12/20 11:26:30 | Picked up JAVA_TOOL_OPTIONS: -Xms32M -Xmx128M

INFO   | jvm 1    | 2017/12/20 11:26:30 | WrapperManager: Initializing...

INFO   | jvm 1    | 2017/12/20 11:26:31 | WrapperSimpleApp:

INFO   | jvm 1    | 2017/12/20 11:26:31 | WrapperSimpleApp Error: Encountered an error running main:

INFO   | jvm 1    | 2017/12/20 11:26:31 | WrapperSimpleApp Error: org.springframework.beans.factory.BeanDefinitionStoreException: Invalid bean definition with name 'vcInventory' defined in class path resource [eam-server-beans.xml]: Could not resolve placeholder 'drs.demandCapacityRatio'

INFO   | jvm 1    | 2017/12/20 11:26:31 | WrapperSimpleApp Error:       at org.springframework.beans.factory.config.PropertyPlaceholderConfigurer.processProperties(PropertyPlaceholderConfigurer.java:268)

INFO   | jvm 1    | 2017/12/20 11:26:31 | WrapperSimpleApp Error:       at org.springframework.beans.factory.config.PropertyResourceConfigurer.postProcessBeanFactory(PropertyResourceConfigurer.java:75)

INFO   | jvm 1    | 2017/12/20 11:26:31 | WrapperSimpleApp Error:       at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:554)

INFO   | jvm 1    | 2017/12/20 11:26:31 | WrapperSimpleApp Error:       at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:528)

INFO   | jvm 1    | 2017/12/20 11:26:31 | WrapperSimpleApp Error:       at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:363)

INFO   | jvm 1    | 2017/12/20 11:26:31 | WrapperSimpleApp Error:       at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)

INFO   | jvm 1    | 2017/12/20 11:26:31 | WrapperSimpleApp Error:       at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:83)

INFO   | jvm 1    | 2017/12/20 11:26:31 | WrapperSimpleApp Error:       at com.vmware.eam.util.Main.main(Main.java:50)

INFO   | jvm 1    | 2017/12/20 11:26:31 | WrapperSimpleApp Error:       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

INFO   | jvm 1    | 2017/12/20 11:26:31 | WrapperSimpleApp Error:       at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

INFO   | jvm 1    | 2017/12/20 11:26:31 | WrapperSimpleApp Error:       at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

INFO   | jvm 1    | 2017/12/20 11:26:31 | WrapperSimpleApp Error:       at java.lang.reflect.Method.invoke(Unknown Source)

INFO   | jvm 1    | 2017/12/20 11:26:31 | WrapperSimpleApp Error:       at org.tanukisoftware.wrapper.WrapperSimpleApp.run(WrapperSimpleApp.java:290)

INFO   | jvm 1    | 2017/12/20 11:26:31 | WrapperSimpleApp Error:       at java.lang.Thread.run(Unknown Source)

STATUS | wrapper  | 2017/12/20 11:26:33 | <-- Wrapper Stopped

Regards,
Chris
VCIX-DCV 2023 | VCIX-NV 2023 | vExpert *** | CCNA R&S
1 Solution

Accepted Solutions
ChrisFD2
VMware Employee
VMware Employee
Jump to solution

I have managed to fix it.

Somehow I had missed drs.demandCapacityRatio=100 off the tail of eam.properties.

Service now started.

Regards,
Chris
VCIX-DCV 2023 | VCIX-NV 2023 | vExpert *** | CCNA R&S

View solution in original post

Reply
0 Kudos
13 Replies
ChrisFD2
VMware Employee
VMware Employee
Jump to solution

I have managed to fix it.

Somehow I had missed drs.demandCapacityRatio=100 off the tail of eam.properties.

Service now started.

Regards,
Chris
VCIX-DCV 2023 | VCIX-NV 2023 | vExpert *** | CCNA R&S
Reply
0 Kudos
Victorious08
Contributor
Contributor
Jump to solution

Hi,

im have trouble same with this case. the eam.properties is blank.

what should i do?

Reply
0 Kudos
ChrisFD2
VMware Employee
VMware Employee
Jump to solution

You need to copy the contents of the file from a vCSA server which has contents in the file, but edit the URL and CM contents so they reflect the correct vlaues.

Regards,
Chris
VCIX-DCV 2023 | VCIX-NV 2023 | vExpert *** | CCNA R&S
Reply
0 Kudos
Victorious08
Contributor
Contributor
Jump to solution

Hallo,

can you help me to send me email/upload. because my VCSA server is corrupt.

Many Thanks

----

Stdout = Starting VMware ESX Agent Manager...

Waiting for VMware ESX Agent Manager.......

WARNING: VMware ESX Agent Manager may have failed to start.

Stderr =

2018-01-12T10:02:07.579Z   {

    "resolution": null,

    "detail": [

        {

            "args": [

                "Command: ['/sbin/service', u'vmware-eam', 'start']\nStderr: "

            ],

            "id": "install.ciscommon.command.errinvoke",

            "localized": "An error occurred while invoking external command : 'Command: ['/sbin/service', u'vmware-eam', 'start']\nStderr: '",

            "translatable": "An error occurred while invoking external command : '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

ERROR:root:Unable to start service vmware-eam, Exception: {

    "resolution": null,

    "detail": [

        {

            "args": [

                "vmware-eam"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'vmware-eam'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

Unable to start service vmware-eam, Exception: {

    "resolution": null,

    "detail": [

        {

            "args": [

                "vmware-eam"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'vmware-eam'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

localhost:~ #

Reply
0 Kudos
Blagovest
Contributor
Contributor
Jump to solution

Hi guys, we have the same issue and I need also the content of eam.properties as ours is blank.

Thank you in advance

Reply
0 Kudos
Andrew_Judge
Contributor
Contributor
Jump to solution

So you come to this forum asking for help, then leave people in the dark? Pay it forward man 'cmon.

For those of you wondering what he's writing, there's a file called eam.properties on the vcenter appliance. It's located in the /etc/vmware-eam/ folder. Its contents configure the startup parameters for eam when startup occurs

# service-control --start vmware-eam

It looks like this (don't just copy this - notice his error had specific notation to drs...)

#########################################################################

# Copyright 2013-2017 VMware, Inc.  All rights reserved. VMware Confidential

#########################################################################

vc.proxy.host=localhost

vc.proxy.port=80

# Hostname or IP of the EAM server

# Fill only if EAM is not running on the same host as VC

eam.host=

# EAM service port used to configure the HTTP connector of the application server.

eam.int.http.port=15005

# Port and scheme configuration which is used by the ESX 6.x hosts to reach EAM Vib

# file server.

eam.ext.port=443

eam.ext.scheme=https

# Port and scheme configuration which is used by the ESX 5.x hosts to reach EAM Vib

# file server.

eam.ext.port.deprecated=80

eam.ext.scheme.deprecated=http

eam.support_linked_clone=true

eam.clear_db_on_startup=false

eam.debug_ref_count=false

eam.recent_event_size=20

# Value is specified in minutes (set to 24h = 1440m)

eam.scan_for_unknown_agent_vms=1440

# The timeout to wait for hostd to restart on a host (set to 5m=300s)

eam.hostd_restart_timeout=300

# The following entries will be added verbatim to the advanced options

# of hosts on which EAM is enabled. All are optional.

Net.DVFilterBindIpAddress=169.254.0.1

Net.TrafficFilterIpAddress=

#The IP for the VSWIF NIC on the dvFilter switch (for ESX classic).

Net.DVFilterVswifIpAddress=169.254.0.2

# Resource bundle configuration

eam.resourcebundle.filename=eam-resourcebundle.jar

# VLSI embedded tcServer configuration

#

tcserver.tmp.dir=/var/tmp/vmware/eam/tomcat

eam.web.root=/usr/lib/vmware-eam/web

# EAM SSL configuration

#

eam.keystore.type=VKS

eam.key.alias=vpxd-extension

eam.keystore.storename=vpxd-extension

# CM configuration

#

cm.url=http://localhost:18090/cm/sdk/?hostid=16a75f54-16e1-4e97-a211-8f4762f8041c

cm.wait.attempts=360

cm.wait.intervalSeconds=5

# SSO configuration

#

sso.wait.attempts=360

sso.wait.intervalSeconds=5

# VC SSL configuration

#

vc.truststore.type=VKS

vc.truststore.storename=TRUSTED_ROOTS

vc.tunnelSdkUri.template=https://##{VC_HOST_NAME}##:8089/sdk/vimService

vc.tunnelSdkUri=https://YOUR.VC.FQDN:8089/sdk/vimService

drs.demandCapacityRatio=100

##### END FILE

you can

# tail -f /var/log/vmware/eam/eam.log

in a separate session while starting the service through

# service-control --start vmware-eam

Watch out for cert errors - that can cause issues as well. I still haven't got mine going, but hopefully this helps someone. I can't upgrade a vSphere environment because of this and I have to...

ChrisFD2
VMware Employee
VMware Employee
Jump to solution

I am on holiday and don’t have access to my lab or work hence I haven’t answered. I hope that’s okay?

Regards,
Chris
VCIX-DCV 2023 | VCIX-NV 2023 | vExpert *** | CCNA R&S
Reply
0 Kudos
Victorious08
Contributor
Contributor
Jump to solution

Hello,

there is the tail-f of log.

2018-01-29T03:01:28.361Z |  INFO | WrapperSimpleAppMain | Main.java | 47 | Initializing Spring from 'standalone.xml' and 'eam.properties'

2018-01-29T03:01:28.392Z |  INFO | WrapperSimpleAppMain | AbstractApplicationContext.java | 412 | Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@37f0c4d2: display name [org.springframework.context.support.ClassPathXmlApplicationContext@37f0c4d2]; startup date [Mon Jan 29 03:01:28 UTC 2018]; root of context hierarchy

2018-01-29T03:01:28.574Z |  INFO | WrapperSimpleAppMain | XmlBeanDefinitionReader.java | 323 | Loading XML bean definitions from class path resource [standalone.xml]

2018-01-29T03:01:28.717Z |  INFO | WrapperSimpleAppMain | XmlBeanDefinitionReader.java | 323 | Loading XML bean definitions from class path resource [eam-server-beans.xml]

2018-01-29T03:01:28.814Z |  INFO | WrapperSimpleAppMain | AbstractApplicationContext.java | 427 | Bean factory for application context [org.springframework.context.support.ClassPathXmlApplicationContext@37f0c4d2]: org.springframework.beans.factory.support.DefaultListableBeanFactory@2b27f9cb

2018-01-29T03:01:28.947Z |  INFO | WrapperSimpleAppMain | PropertiesLoaderSupport.java | 178 | Loading properties file from class path resource [eam.properties]

2018-01-29T03:01:28.952Z |  INFO | WrapperSimpleAppMain | DefaultSingletonBeanRegistry.java | 396 | Destroying singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@2b27f9cb: defining beans [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0,commonMapStore,vibServices,esxAgentMgr,vcTrustStore,eamKeyStore,eamKeyAccess,vimAndVumConnectionExecutor,vpxdConnection,vumConnection,vcOptions,serverAddr,serverAddrDepr,serverUrl,serverUrlDepr,vcListener,l10n,httpNfcLeaseTimeoutHandler,nameManager,vcDatastoreManager,vcEventManager,vcKeyValueStore,messageSource,vibServer,vibServerDepr,vibPackageDepot,vibPackageDepotDepr,secureHostsStartVersion,vibPackageDepots,vcInventory,springContext,scanForUnknownAgentVms,cmClient,stsCertManager,scheduler,heathStatusTaskScheduler,vmodlContext,versionFinder,clientAuthConnectionExecutor,clientAuthConnection,clientAuthenticator,localizationFilter,firstMobStep,mobConnectionExecutor,mobConnection,mobAuthenticator,localClient,localCorrelator,localFilterStartStep,localCompletionStep,sessionManager,dispatcher,adapterServer,eamVersion,bodyHandler,eamVersion2,bodyHandlerVersion2,eamVersion2_5,bodyHandlerVersion2_5,eamVersion3,bodyHandlerVersion3,eamVersion6,bodyHandlerVersion6,internalEamVersion1,internalBodyHandlerVersion1,internalEamVersion2,internalBodyHandlerVersion2,internalEamVersion2_5,internalBodyHandlerVersion2_5,internalEamVersion3,internalBodyHandlerVersion3,internalEamVersion6,internalBodyHandlerVersion6,httpResponseCorrelator,firstStep,validationStartStep,completionStep,filterStartStep,invocationStep,continuerStep,rejectedHandler,executor,httpServerEndpoint,listener,mob,jspServlet,fileServlet,health,managementUi,cmHealthServlet,cmResourceBundleServlet,vmomiService,mobService,jspService,filesService,healthService,managementService,vibService,vibdService,cmHealthService,cmResourceBundleService,httpServer]; root of factory hierarchy

pastedImage_1.png

the services wont up.

Reply
0 Kudos
ChrisFD2
VMware Employee
VMware Employee
Jump to solution

What's the contents of your /etc/vmware-eam/eam.properties file? From your 'ls' it looks small. I can't share my file details with you as it looks quite different in 6.5, I presume EAM has changed somewhat.

Use what Andrew Judge posted and change it to match your environment. I *think* the only lines you need to change are:

cm.url=http://localhost:18090/cm/sdk/?hostid=16a75f54-16e1-4e97-a211-8f4762f8041c

vc.tunnelSdkUri.template=https://##{VC_HOST_NAME}##:8089/sdk/vimService

vc.tunnelSdkUri=https://YOUR.VC.FQDN:8089/sdk/vimService

Obviously the cm.url value should match your vCSA server. I can't quite remember where I got my value from.

Do you have more than one vCSA server? What is your setup? Do you have backups?

Regards,
Chris
VCIX-DCV 2023 | VCIX-NV 2023 | vExpert *** | CCNA R&S
Reply
0 Kudos
munishpalmakhij
Contributor
Contributor
Jump to solution

Hey Andrew,

We also ran into the same issue. How do you obtain the hostid for cm.url ?

I tried the uuid of the VC VM however I am still unable to start the vmware-eam service

Regards

-Munish

Reply
0 Kudos
ChrisFD2
VMware Employee
VMware Employee
Jump to solution

Do you have a vCSA where thee am service starts?

Regards,
Chris
VCIX-DCV 2023 | VCIX-NV 2023 | vExpert *** | CCNA R&S
Reply
0 Kudos
dertz
Contributor
Contributor
Jump to solution

Munish:

Ever get a response on the hostid?  Trying to rebuild file as mine got clobbered somehow and just need this piece of info.

Thanks,

Doug

Reply
0 Kudos
dertz
Contributor
Contributor
Jump to solution

figure out how to get the hostid

run command:   cat /etc/vmware/install-defaults/sca.hostid

THis will get you hostid.  There is a  good article on this issue at VMware Knowledge Base

Reply
0 Kudos