rinku02bansal
Contributor
Contributor

We have setup VMware appliance in our setup but after changing some configuration in inventory.cfg , Vmware-vpxd services will not starting even after reverting the changes. Attaching logs below. kindly help us ASAP.

Waiting for the embedded database to start up: success

Verifying EULA acceptance: success

Executing pre-startup scripts...

Updating the vCenter endpoint in the Lookup Service.

Intializing registration provider...

Getting SSL certificates for https://192.168.20.100:7444/lookupservice/sdk

Service with name 'vpxd-localhost.localdom-543843fd-d8da-4c7f-9bd7-8acd95edeb60' and ID 'local:9c27a9ee-0ca0-42e7-8767-7704f847d240' was updated.

Return code is: Success

Shutting down ldap-server..done

Starting ldap-server..done

Starting vmware-vpxd: success

Waiting for vpxd to initialize: .failed     <-- this is going to be fail every time.

localhost:~ #

On System log , We will get :-

localhost:~ # cat /var/log/messages

2017-06-05T10:13:18+00:00 localhost watchdog-vpxd: PID file /var/run/vmware/watchdog-vpxd.PID does not exist

2017-06-05T10:13:18+00:00 localhost watchdog-vpxd: Unable to terminate watchdog: No running watchdog process for vpxd

2017-06-05T10:13:19+00:00 localhost watchdog-vpxd: PID file /var/run/vmware/watchdog-vpxd.PID does not exist

2017-06-05T10:13:19+00:00 localhost watchdog-vpxd: Unable to terminate watchdog: No running watchdog process for vpxd

2017-06-05T10:13:27+00:00 localhost watchdog-vpxd: [12418] Begin '/usr/sbin/vpxd', min-uptime = 30, max-quick-failures = 5, max-total-failures = 1000000, bg_pid_file = ''

2017-06-05T10:13:27+00:00 localhost watchdog-vpxd: Executing '/usr/sbin/vpxd'

2017-06-05T10:13:27+00:00 localhost watchdog-vpxd: '/usr/sbin/vpxd' exited after 0 seconds (quick failure 1) 1

2017-06-05T10:13:27+00:00 localhost watchdog-vpxd: Executing '/usr/sbin/vpxd'

2017-06-05T10:13:28+00:00 localhost watchdog-vpxd: '/usr/sbin/vpxd' exited after 1 seconds (quick failure 2) 1

2017-06-05T10:13:28+00:00 localhost watchdog-vpxd: Executing '/usr/sbin/vpxd'

2017-06-05T10:13:28+00:00 localhost watchdog-vpxd: '/usr/sbin/vpxd' exited after 0 seconds (quick failure 3) 1

2017-06-05T10:13:28+00:00 localhost watchdog-vpxd: Executing '/usr/sbin/vpxd'

2017-06-05T10:13:28+00:00 localhost watchdog-vpxd: '/usr/sbin/vpxd' exited after 0 seconds (quick failure 4) 1

2017-06-05T10:13:28+00:00 localhost watchdog-vpxd: Executing '/usr/sbin/vpxd'

2017-06-05T10:13:29+00:00 localhost watchdog-vpxd: '/usr/sbin/vpxd' exited after 1 seconds (quick failure 5) 1

2017-06-05T10:13:29+00:00 localhost watchdog-vpxd: Executing '/usr/sbin/vpxd'

2017-06-05T10:13:29+00:00 localhost watchdog-vpxd: '/usr/sbin/vpxd' exited after 0 seconds (quick failure 6) 1

2017-06-05T10:13:29+00:00 localhost watchdog-vpxd: End '/usr/sbin/vpxd', failure limit reached

2017-06-05T10:14:13+00:00 localhost watchdog-vpxd: PID file /var/run/vmware/watchdog-vpxd.PID does not exist

2017-06-05T10:14:13+00:00 localhost watchdog-vpxd: Unable to terminate watchdog: No running watchdog process for vpxd

2017-06-05T10:14:14+00:00 localhost watchdog-vpxd: PID file /var/run/vmware/watchdog-vpxd.PID does not exist

2017-06-05T10:14:14+00:00 localhost watchdog-vpxd: Unable to terminate watchdog: No running watchdog process for vpxd

2017-06-05T10:15:56+00:00 localhost watchdog-vpxd: [4936] Begin '/usr/sbin/vpxd', min-uptime = 30, max-quick-failures = 5, max-total-failures = 1000000, bg_pid_file = ''

2017-06-05T10:15:56+00:00 localhost watchdog-vpxd: Executing '/usr/sbin/vpxd'

2017-06-05T10:15:56+00:00 localhost kernel: [   82.191937] warning: process `vpxd' used the deprecated sysctl system call with 1.1.

2017-06-05T10:15:56+00:00 localhost watchdog-vpxd: '/usr/sbin/vpxd' exited after 0 seconds (quick failure 1) 1

2017-06-05T10:15:56+00:00 localhost watchdog-vpxd: Executing '/usr/sbin/vpxd'

2017-06-05T10:15:57+00:00 localhost watchdog-vpxd: '/usr/sbin/vpxd' exited after 1 seconds (quick failure 2) 1

2017-06-05T10:15:57+00:00 localhost watchdog-vpxd: Executing '/usr/sbin/vpxd'

2017-06-05T10:15:57+00:00 localhost watchdog-vpxd: '/usr/sbin/vpxd' exited after 0 seconds (quick failure 3) 1

2017-06-05T10:15:57+00:00 localhost watchdog-vpxd: Executing '/usr/sbin/vpxd'

2017-06-05T10:15:57+00:00 localhost watchdog-vpxd: '/usr/sbin/vpxd' exited after 0 seconds (quick failure 4) 1

2017-06-05T10:15:57+00:00 localhost watchdog-vpxd: Executing '/usr/sbin/vpxd'

2017-06-05T10:15:58+00:00 localhost watchdog-vpxd: '/usr/sbin/vpxd' exited after 1 seconds (quick failure 5) 1

2017-06-05T10:15:58+00:00 localhost watchdog-vpxd: Executing '/usr/sbin/vpxd'

2017-06-05T10:15:58+00:00 localhost watchdog-vpxd: '/usr/sbin/vpxd' exited after 0 seconds (quick failure 6) 1

2017-06-05T10:15:58+00:00 localhost watchdog-vpxd: End '/usr/sbin/vpxd', failure limit reached

2017-06-05T10:20:01+00:00 localhost /usr/sbin/cron[5640]: (root) CMD ( test -x /usr/sbin/vpxd_periodic && /usr/sbin/vpxd_periodic >/dev/null 2>&1)

On vpxd.log, I also get some error.

localhost:/var/log/vmware/vpx # tailf -n 100 vpxd-171.log

Section for VMware VirtualCenter, pid=27456, version=5.5.0, build=2001466, option=Release

------ In-memory logs start --------

mem> 2017-06-06T04:14:20.657Z [7F9131494740 verbose 'vpxdvpxdSignal'] Signal handler for signal 1 is installed

mem> 2017-06-06T04:14:20.657Z [7F9131494740 verbose 'vpxdvpxdSignal'] Signal handler for signal 2 is installed

mem> 2017-06-06T04:14:20.657Z [7F9131494740 verbose 'vpxdvpxdSignal'] Signal handler for signal 15 is installed

mem> 2017-06-06T04:14:20.657Z [7F9131494740 verbose 'vpxdvpxdSignal'] Signal handler for signal 10 is installed

mem> 2017-06-06T04:14:20.657Z [7F9131494740 info 'Hooks'] Hooks Initialized

------ In-memory logs end   --------

2017-06-06T04:14:20.660Z [7F9131494740 info 'Default'] Logging uses fast path: true

2017-06-06T04:14:20.660Z [7F9131494740 info 'Default'] Handling bora/lib logs with VmaCore facilities

2017-06-06T04:14:20.660Z [7F9131494740 info 'Default'] Initialized channel manager

2017-06-06T04:14:20.663Z [7F9131494740 info 'Default'] Current working directory: /storage/log/vmware/vpx

2017-06-06T04:14:20.663Z [7F9131494740 info 'ThreadPool'] Thread pool on asio: Min Io, Max Io, Min Task, Max Task, Max Concurency: 2, 200, 2, 90, 2147483647

2017-06-06T04:14:20.663Z [7F9131494740 info 'ThreadPool'] Thread enlisted

2017-06-06T04:14:20.663Z [7F9131494740 info 'Default'] Log path: /var/log/vmware/vpx

2017-06-06T04:14:20.663Z [7F9131494740 info 'Default'] Initializing SSL

2017-06-06T04:14:20.663Z [7F91278A4700 info 'ThreadPool'] Thread enlisted

2017-06-06T04:14:20.663Z [7F91279A6700 info 'ThreadPool'] Thread enlisted

2017-06-06T04:14:20.663Z [7F9127823700 info 'ThreadPool'] Thread enlisted

2017-06-06T04:14:20.663Z [7F9127925700 info 'ThreadPool'] Thread enlisted

2017-06-06T04:14:20.664Z [7F9131494740 info 'Default'] Vmacore::InitSSL: handshakeTimeoutUs = 120000000

2017-06-06T04:14:20.665Z [7F91277A2700 info 'ThreadPool'] Thread enlisted

2017-06-06T04:14:20.665Z [7F9131494740 info 'vpxdvpxdDaemon'] Changed working directory to /var/log/vmware/vpx

2017-06-06T04:14:20.666Z [7F9131494740 error 'vpxdvpxdDaemon'] Unhandled exception:

--> not well-formed (invalid token)

2017-06-06T04:14:20.667Z [7F9131494740 error 'vpxdvpxdDaemon'] Backtrace:

--> backtrace[00] rip 00007f912b3f9014 Vmacore::System::Stacktrace::CaptureWork(unsigned int)

--> backtrace[01] rip 00007f912b2e3132 Vmacore::System::SystemFactoryImpl::CreateQuickBacktrace(Vmacore::Ref<Vmacore::System::Backtrace>&)

--> backtrace[02] rip 00007f912b2448a5 Vmacore::Throwable::Throwable(std::string const&)

--> backtrace[03] rip 00007f912b28b566 /usr/lib/vmware-vpx/libvmacore.so(+0x285566) [0x7f912b28b566]

--> backtrace[04] rip 00007f912b23e85d /usr/lib/vmware-vpx/libvmacore.so(+0x23885d) [0x7f912b23e85d]

--> backtrace[05] rip 00007f9132abfd07 /usr/lib/vmware-vpx/vpxd(+0x14aad07) [0x7f9132abfd07]

--> backtrace[06] rip 00007f9132a3f0c5 /usr/lib/vmware-vpx/vpxd(+0x142a0c5) [0x7f9132a3f0c5]

--> backtrace[07] rip 00007f91324c3ca3 /usr/lib/vmware-vpx/vpxd(+0xeaeca3) [0x7f91324c3ca3]

--> backtrace[08] rip 00007f91324a6c2e /usr/lib/vmware-vpx/vpxd(+0xe91c2e) [0x7f91324a6c2e]

--> backtrace[09] rip 00007f9129369c16 /lib64/libc.so.6(__libc_start_main+0xe6) [0x7f9129369c16]

--> backtrace[10] rip 00007f91324a5e89 /usr/lib/vmware-vpx/vpxd(+0xe90e89) [0x7f91324a5e89]

I have gone through the above solutions for storage space full but i checked on our setup its not there.

localhost:/var/log/vmware/vpx # df -h

Filesystem      Size  Used Avail Use% Mounted on

/dev/sda3       9.8G  3.7G  5.6G  40% /

udev            4.0G  104K  4.0G   1% /dev

tmpfs           4.0G     0  4.0G   0% /dev/shm

/dev/sda1       128M   21M  101M  18% /boot

/dev/sdb1        20G  173M   19G   1% /storage/core

/dev/sdb2        20G  305M   19G   2% /storage/log

/dev/sdb3        60G  1.7G   55G   4% /storage/db

localhost:/var/log/vmware/vpx # df -ih

Filesystem     Inodes IUsed IFree IUse% Mounted on

/dev/sda3        632K   60K  573K   10% /

udev                0     0     0     - /dev

tmpfs           1001K     2 1001K    1% /dev/shm

/dev/sda1         34K    40   34K    1% /boot

/dev/sdb1        1.3M    12  1.3M    1% /storage/core

/dev/sdb2        1.3M   293  1.3M    1% /storage/log

/dev/sdb3        3.8M  3.7K  3.8M    1% /storage/db

localhost:/var/log/vmware/vpx #

Reply
0 Kudos