VMware Cloud Community
Per_W
Contributor
Contributor
Jump to solution

VMware Inventory Service dont start - vCenter v6.0 u3

vCenter v6.0 u3

So I ran in to some problems when I came back to work after my vacation. vCenter Servers disk space was filled to max, because of not having changed the root password for awhile, so that the rotation of the logs didn't work and therefore filled my disks.

I free some space by following KB2069041 - https://kb.vmware.com/s/article/2069041

     Change the root password and then went to

    /var/log/audit

     > audit.log

Restarted the vCenter and it work fine until the service Inventory Service comes along. This service woun't start.

I could here log in to the shell and check some more.

I can see that the rotation of the logs works

     ls -l /var/spool/cron/lastrun/

To clear up even more log files I found this, don't know if its any good, but here it is.

http://easi123.4realit.nl/2016/07/safely-cleaning-up-log-files-vcentervrealize/

     find /storage/log/ -mount -type f -mtime +1 -exec echo {} \; -exec truncate -cs 0 {} \; 2>&1 | tee /tmp/files_truncated.txt

Tried to start the service again manually

     service-control --start --all

But when it comes down to Inventory Service this error appears and the startup seq. ends.

****************************

Stdout = Starting VMware Inventory Service...

Waiting for VMware Inventory Service...............

WARNING: VMware Inventory Service may have failed to start.

Stderr =

2018-07-25T06:05:31.155Z   {

    "resolution": null,

    "detail": [

        {

            "args": [

                "Command: ['/sbin/service', u'vmware-invsvc', 'start']\nStderr: "

            ],

            "id": "install.ciscommon.command.errinvoke",

            "localized": "An error occurred while invoking external command : 'Command: ['/sbin/service', u'vmware-invsvc', 'start']\nStderr: '",

            "translatable": "An error occurred while invoking external command : '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

ERROR:root:Unable to start service vmware-invsvc, Exception: {

    "resolution": null,

    "detail": [

        {

            "args": [

                "vmware-invsvc"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'vmware-invsvc'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

Unable to start service vmware-invsvc, Exception: {

    "resolution": null,

    "detail": [

        {

            "args": [

                "vmware-invsvc"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'vmware-invsvc'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

****************************

I have tried to clear the Inventory by following KB2146248. This made no diffrens.

So I backed to the snapshot I did before trying KB2146248.

So now I ran out of ide's, anyone else have any suggestion what to do now

How to get the Inventory Service to start again.

/ Per W

1 Solution

Accepted Solutions
Vijay2027
Expert
Expert
Jump to solution

Encountered similar issue recently.

Post clearing audit.log inventory service failed to start. Verified inventory service logs and resolved the issue by following the KB:

VMware Knowledge Base

Make sure you take a snapshot before you proceed with the troubleshooting.

View solution in original post

5 Replies
Vijay2027
Expert
Expert
Jump to solution

Encountered similar issue recently.

Post clearing audit.log inventory service failed to start. Verified inventory service logs and resolved the issue by following the KB:

VMware Knowledge Base

Make sure you take a snapshot before you proceed with the troubleshooting.

Per_W
Contributor
Contributor
Jump to solution

Thanks! This worked for me.

The Inventory Service is now starting. Yeah!

But it only took me to the next service "vmware-eam" or VMware ESX Agent Manager, is not starting.

So the troubleshooting continues...

Reply
0 Kudos
Vijay2027
Expert
Expert
Jump to solution

Look at this community thread:

ESX Agent Manager - vmware-eam will not start

You will need to change below entries:

cm.url=http://localhost:18090/cm/sdk/?hostid=0a12dfe2-b3b7-4eb6-9737-1ec5a8e770ec

vc.tunnelSdkUri=<vCenter_FQDN>:8089/sdk/vimService

You will have to paste the hostid from cat /etc/vmware/install-defaults/sca.hostid:

Reply
0 Kudos
Vijay2027
Expert
Expert
Jump to solution

Were you able to fix eam service?

Reply
0 Kudos
Per_W
Contributor
Contributor
Jump to solution

So the eam service didn't start, read some articles about the problem, then checked my file /etc/vmware-eam/eam.properties

It was completely blanked out except for one row. This might occur when the disk is full I read on some page.

Started two sessions to the vCenter and ran in one to see the live logs.

     tail -f /var/log/vmware/eam/eam.log

Then started the service in the next.

     service-control --start vmware-eam

Got the following in the log:

Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanDefinitionStoreException: Invalid bean definition with name 'esxAgentMgr' defined in class path resource [eam-server-beans.xml]: Could not resolve placeholder 'eam.hostd_restart_timeout' in string value "${eam.hostd_restart_timeout}"; nested exception is java.lang.IllegalArgumentException: Could not resolve placeholder 'eam.hostd_restart_timeout' in string value "${eam.hostd_restart_timeout}"

Searched the internet for awhile and found a KB.

I Followed KB2148582, https://kb.vmware.com/s/article/2148582, to solve the problem.

I downloaded the attached file of the KB and ran the script to create a new eam.properties file.

After completing the file I tried to start the service again.

     service-control --start vmware-eam

It worked.

All services on my vCenter is now starting!

Thanks for the help.

Removed all my error-searching-snapshots and did a fresh backup of the appliance and the PostgreSQL database.

Hopefully this error woun't occur again.

The root of the problem was that the root password hadn't been changed for a long while. This resulted in the log-files to grow, and never rotate.

So my tip is to change the root password once a year or just log in to your vCenter

     https://<vCenter-address>:5480

and then go to administation and change the "Password expiry settings" to -

/ Per

Reply
0 Kudos