VMware Cloud Community
jasekera
Contributor
Contributor

Service vmware-vpxd won't start after appliance restart

Hi guys,

after I reboot/restart my vCenter appliance (virtual machine) vmware-vpxd won't start and fail within initialization (everytime). After 3 days of researching I have a workaround:

1. ssh into vCenter appliance and run command: ps -ef | grep vpxd

2. here I can see, that there is something locked out - connected with vmware-vpxd service (flock -on ....)

3. run command service-control --stop --all

4. then I kill all processes (with command kill), which are listed as output from command  ps -ef | grep vpxd (connected with vmware-vpxd service)

5. run command service-control --start --all

6. when services are starting I have to type password, which is stored in (/etc/vmware-vpx/embedded_db.cfg) - postgres database password

After this process everything is fine and working, but I don't have an idea how to fix this problem permament. I don't want to do this workaround after every restart. My vCenter version is 6.0.0U3f and my idea was to upgrade to version 6.5.0 (but actually not possible). Any idea, how to fix this problem without migrating to 6.5.0?

Thank you for your answer.

Jakub

Reply
0 Kudos
10 Replies
daphnissov
Immortal
Immortal

The first step is to look at (and/or post) the log file to see what's actually being logged. If you have the ability to open an SR with VMware, you may want to get one of those started as well.

Reply
0 Kudos
jasekera
Contributor
Contributor

Dear daphnissov,

yeah of course. I will look up into logs and I'll post there. Should I post vpxd.log and what else?

Jakub

Reply
0 Kudos
daphnissov
Immortal
Immortal

I'd recommend you look at your own logs first (start with vpxa.log) and see what you can find. Use some Google skills to see if there are known KBs, and search the communities here. One should make every attempt to help themselves (and potentially learn a great deal in the process) before asking for help.

Reply
0 Kudos
jasekera
Contributor
Contributor

Dear daphnissov​,

I understand and I took maybe 3-4 days (google, known KBs, the communities here) and I don't find any the solution for my problem, unfortunately. Then I opened vmware-vpxd script in text editor, debug code and find out, that there can be a problem with locking. I think, that can be a very specific strange problem, so for that reason maybe the support of vmware is the best way, right?

Jakub

Reply
0 Kudos
daphnissov
Immortal
Immortal

But you haven't even looked at the log yet, so why are you even doing that? Look at the log first at the entries created after you attempt to start the process and see what it says.

Reply
0 Kudos
jasekera
Contributor
Contributor

Sorry for misunderstandings, of course I looked at the logs, and so many times Smiley Happy, but I can't find anything, which can solve my problem. It's bunch of logs and a lot of dependencies and sometimes it can be specific and so hard to find out, where is a problem.

Jakub

Reply
0 Kudos
daphnissov
Immortal
Immortal

I'd recommend restarting your vCSA, then attaching the vpx log to a response. Please do not copy/paste raw text into your response.

Reply
0 Kudos
jasekera
Contributor
Contributor

Hi guys,

after one month issue occurs again. Below I attached screenshots.

Screen Shot 2018-08-19 at 23.03.13 1.png

As you can see there is a problem with a flock -on ..., which I mentioned in above post.

Screen Shot 2018-08-21 at 08.37.38.png

As you can see there, the process is waiting for the DB to start up, but I waited and nothing happend.

Screen Shot 2018-08-19 at 23.04.27.png

There is vmware-vpxd.log, in vpxd.log there is nothing logged about this problem, only logs before restart VMs, when it was working correctly.

Then I used my workaround mentioned in my first post and everything was ok.

Any ideas?

Jakub

Reply
0 Kudos
Vijay2027
Expert
Expert

Can you check if enough is available for the mount points.

# df -Th

Reply
0 Kudos
jasekera
Contributor
Contributor

Screen Shot 2018-08-21 at 09.19.29.png

Of course it is, as I mentioned in my first post, when I manually stop all services, kill the process manually, start all services and type password for database everything is fine. Maybe (I don't know why), when it is starting automatically after restart, the process want a database password, but nobody doesn't provide that and process locked out? I don't have any idea, where can be a problem and I was searching for any solution, but without success.

Thank you in advance.

Jakub

Reply
0 Kudos