VMware Cloud Community
111zzz888
Contributor
Contributor
Jump to solution

"A general system error has occurred: Timed out waiting for vpxa to start".

I am trying to re-add a disconnected cluster. There was an issue with the license, which has been recently resolved and the new license has been applied.

I have had a quick look through the existing threads and cannot find a comparable issue.

Things of note:

Specifically the vDRS has been turned off on the cluster, and at the moment it is not required (single host).

The IP address of the vcenter has also changed, it is very likely either of the license change or the recent IP address change have caused the issue (it needed to be moved to another network).

I have updated the vpxa.conf with the new IP address, and have set the port to no avail.

vCenter host has also been rebooted.

vpxa.log does not seem to be updating.

vCenter Version: 6.50.20000 Build 9451637

6.5.0 Build Number 8294253

There is a firewall between the 2 however the firewall is definitely not blocking traffic between the two IP addresses.

Here is the output of the syslog:

2018-12-11T15:57:40Z watchdog-vpxa: [8456549] Begin '/usr/lib/vmware/vpxa/bin/vpxa ++min=0,swapscope=system -D /etc/vmware/vpxa', min-uptime = 60, max-quick-failures = 5, max-total-failures = 1000000, bg_pid_file = '', reboot-flag = '0'

2018-12-11T15:57:40Z watchdog-vpxa: Executing '/usr/lib/vmware/vpxa/bin/vpxa ++min=0,swapscope=system -D /etc/vmware/vpxa'

2018-12-11T15:57:45Z backup.sh.8456580: Locking esx.conf

2018-12-11T15:57:45Z backup.sh.8456580: Creating archive

2018-12-11T15:57:45Z backup.sh.8456580: Unlocking esx.conf

2018-12-11T15:57:50Z watchdog-vpxa: '/usr/lib/vmware/vpxa/bin/vpxa ++min=0,swapscope=system -D /etc/vmware/vpxa' exited after 10 seconds (quick failure 1) 134

2018-12-11T15:57:50Z watchdog-vpxa: Executing '/usr/lib/vmware/vpxa/bin/vpxa ++min=0,swapscope=system -D /etc/vmware/vpxa'

2018-12-11T15:57:54Z smad[8063535]: Failed: no available HP iLO channels

2018-12-11T15:57:54Z smad[8063535]: Failed bmc_init(). Exit smad.

2018-12-11T15:58:01Z watchdog-vpxa: '/usr/lib/vmware/vpxa/bin/vpxa ++min=0,swapscope=system -D /etc/vmware/vpxa' exited after 11 seconds (quick failure 2) 134

2018-12-11T15:58:01Z watchdog-vpxa: Executing '/usr/lib/vmware/vpxa/bin/vpxa ++min=0,swapscope=system -D /etc/vmware/vpxa'

2018-12-11T15:58:09Z smad[100962]: Failed: no available HP iLO channels

2018-12-11T15:58:09Z smad[100962]: Failed bmc_init(). Exit smad.

2018-12-11T15:58:12Z watchdog-vpxa: '/usr/lib/vmware/vpxa/bin/vpxa ++min=0,swapscope=system -D /etc/vmware/vpxa' exited after 11 seconds (quick failure 3) 134

2018-12-11T15:58:12Z watchdog-vpxa: Executing '/usr/lib/vmware/vpxa/bin/vpxa ++min=0,swapscope=system -D /etc/vmware/vpxa'

2018-12-11T15:58:23Z watchdog-vpxa: '/usr/lib/vmware/vpxa/bin/vpxa ++min=0,swapscope=system -D /etc/vmware/vpxa' exited after 11 seconds (quick failure 4) 134

2018-12-11T15:58:23Z watchdog-vpxa: Executing '/usr/lib/vmware/vpxa/bin/vpxa ++min=0,swapscope=system -D /etc/vmware/vpxa'

2018-12-11T15:58:24Z smad[8456832]: Failed: no available HP iLO channels

2018-12-11T15:58:24Z smad[8456832]: Failed bmc_init(). Exit smad.

2018-12-11T15:58:34Z watchdog-vpxa: '/usr/lib/vmware/vpxa/bin/vpxa ++min=0,swapscope=system -D /etc/vmware/vpxa' exited after 11 seconds (quick failure 5) 134

2018-12-11T15:58:34Z watchdog-vpxa: Executing '/usr/lib/vmware/vpxa/bin/vpxa ++min=0,swapscope=system -D /etc/vmware/vpxa'

2018-12-11T15:58:39Z smad[8456846]: Failed: no available HP iLO channels

2018-12-11T15:58:39Z smad[8456846]: Failed bmc_init(). Exit smad.

2018-12-11T15:58:44Z watchdog-vpxa: '/usr/lib/vmware/vpxa/bin/vpxa ++min=0,swapscope=system -D /etc/vmware/vpxa' exited after 10 seconds (quick failure 6) 134

2018-12-11T15:58:44Z watchdog-vpxa: End '/usr/lib/vmware/vpxa/bin/vpxa ++min=0,swapscope=system -D /etc/vmware/vpxa', failure limit reached

vpxa.conf (IP addresses obfuscated)

<config>

  <dvs>

    <portSyncBatchLimit>100</portSyncBatchLimit>

  </dvs>

  <httpNfc>

    <accessMode>proxyAuto</accessMode>

    <enabled>true</enabled>

  </httpNfc>

  <level id="SoapAdapter.HTTPService">

    <logLevel>info</logLevel>

    <logName>SoapAdapter.HTTPService</logName>

  </level>

  <level id="SoapAdapter.HTTPService.HttpConnection">

    <logLevel>info</logLevel>

    <logName>SoapAdapter.HTTPService.HttpConnection</logName>

  </level>

  <log>

    <level>verbose</level>

    <maxFileNum>10</maxFileNum>

    <maxFileSize>1048576</maxFileSize>

    <memoryLevel>verbose</memoryLevel>

    <outputToConsole>false</outputToConsole>

    <outputToFiles>false</outputToFiles>

    <outputToSyslog>true</outputToSyslog>

    <syslog>

      <facility>local4</facility>

      <ident>Vpxa</ident>

      <logHeaderFile>/var/run/vmware/vpxaLogHeader.txt</logHeaderFile>

    </syslog>

  </log>

  <nfc>

    <loglevel>error</loglevel>

  </nfc>

  <task>

    <completedMaxEntries>1000</completedMaxEntries>

    <maxThreads>98</maxThreads>

    <minCompletedLifetime>120</minCompletedLifetime>

  </task>

  <trace>

    <mutex>

      <profiledMutexes>InvtLock</profiledMutexes>

    </mutex>

    <vmomi>

      <calls>false</calls>

    </vmomi>

  </trace>

  <vmacore>

    <http>

      <defaultClientPoolConnectionsPerServer>300</defaultClientPoolConnectionsPerServer>

    </http>

    <soap>

      <sessionTimeout>1440</sessionTimeout>

    </soap>

    <ssl>

      <doVersionCheck>false</doVersionCheck>

    </ssl>

    <threadPool>

      <IoMax>9</IoMax>

      <TaskMax>4</TaskMax>

      <ThreadStackSizeKb>128</ThreadStackSizeKb>

      <threadNamePrefix>vpxa</threadNamePrefix>

    </threadPool>

  </vmacore>

  <vpxa>

    <bundleVersion>1000000</bundleVersion>

    <datastorePrincipal>root</datastorePrincipal>

    <hostIp>X.X.X.X</hostIp>

    <hostKey/>

    <hostPort>443</hostPort>

    <licenseExpiryNotificationThreshold>15</licenseExpiryNotificationThreshold>

    <memoryCheckerTimeInSecs>30</memoryCheckerTimeInSecs>

    <serverIp>Y.Y.Y.Y</serverIP>

    <serverPort>902</serverPort>

  </vpxa>

  <workingDir>/var/log/vmware/vpx</workingDir>

Update:

vmkernel.log:

2018-12-11T20:58:11.027Z cpu13:8510156)User: 3089: vpxa: wantCoreDump:vpxa signal:6 exitCode:0 coredump:enabled

2018-12-11T20:58:11.203Z cpu4:8510156)UserDump: 3024: vpxa: Dumping cartel 8510156 (from world 8510156) to file /var/core/vpxa-zdump.003 ...

2018-12-11T20:58:21.532Z cpu16:8510156)UserDump: 3172: vpxa: Userworld(vpxa) coredump complete.

Reply
0 Kudos
1 Solution

Accepted Solutions
111zzz888
Contributor
Contributor
Jump to solution

Okay I ended up having to remove the host from the inventory, manually removing all mention of the previous vcenter:

<vpxa>

    <bundleVersion>1000000</bundleVersion>

    <datastorePrincipal>root</datastorePrincipal>

    <hostIp>X.X.X.X</hostIp>

    <hostKey/>

    <hostPort>443</hostPort>

    <licenseExpiryNotificationThreshold>15</licenseExpiryNotificationThreshold>

    <memoryCheckerTimeInSecs>30</memoryCheckerTimeInSecs>

    <serverIp>Y.Y.Y.Y</serverIP>

    <serverPort>902</serverPort>

  </vpxa>

take

<serverIp>Y.Y.Y.Y</serverIP>

replace with

<serverIp/>

take

<serverPort>902</serverPort>

replace with

<serverPort/>

Re-add the host to the cluster and the issue is resolved.

View solution in original post

Reply
0 Kudos
8 Replies
daphnissov
Immortal
Immortal
Jump to solution

If you've changed the IP address of vCenter and vCenter was originally installed with that in place of a name, this has likely caused your issue. You can check if this is the case, and if it is, your only option is to redeploy.

Reply
0 Kudos
111zzz888
Contributor
Contributor
Jump to solution

no the VCA was installed with a fqdn, not an IP address. The fqdn is still the same and doesnt need changing.

Reply
0 Kudos
IRIX201110141
Champion
Champion
Jump to solution

You can "un-join" the ESXi Host from the vCenter. If you use the Hostclient and connect to https://esxi/ui you can select a "disconnect from vCenter" on that given host. Now you can try again a Add-Host on vCenter again.

There is no vDS used right?

Edit: Within vCenter there are Adv. vCenter settings and one of these (Email, Statistic Level, vCenter ID) is the vCenter IP. Please dont forget to change these as well after changing the IP of the VCSA.

Regards,

Joerg

Reply
0 Kudos
111zzz888
Contributor
Contributor
Jump to solution

No vDS, correct.

All attempts to connect the vCenter cause a vpxa core dump, though I am having trouble finding the root cause of this.

Edit: Checking those advanced settings now - sure they are updated but doublechecking now.

Edit 2: Those settings are all correct and the vpxa issue persists.

Reply
0 Kudos
111zzz888
Contributor
Contributor
Jump to solution

Okay I ended up having to remove the host from the inventory, manually removing all mention of the previous vcenter:

<vpxa>

    <bundleVersion>1000000</bundleVersion>

    <datastorePrincipal>root</datastorePrincipal>

    <hostIp>X.X.X.X</hostIp>

    <hostKey/>

    <hostPort>443</hostPort>

    <licenseExpiryNotificationThreshold>15</licenseExpiryNotificationThreshold>

    <memoryCheckerTimeInSecs>30</memoryCheckerTimeInSecs>

    <serverIp>Y.Y.Y.Y</serverIP>

    <serverPort>902</serverPort>

  </vpxa>

take

<serverIp>Y.Y.Y.Y</serverIP>

replace with

<serverIp/>

take

<serverPort>902</serverPort>

replace with

<serverPort/>

Re-add the host to the cluster and the issue is resolved.

Reply
0 Kudos
Kenneth753
Contributor
Contributor
Jump to solution

I trying to add the hosts to the vCenter's data center via Add host, it gets the credentials, displays the name of VMs on the Host Summary page correctly and after clicking on Finish in the Ready to complete screen, times out after some time saying A general system error occurred: Timed out waiting for vpxa to start. I have created DNS entries and PTR entries on my domain server. Time on the vCenter machine and the ESXi host are correct. Please help mybkexperience.

Reply
0 Kudos
111zzz888
Contributor
Contributor
Jump to solution

Please ensure that you have followed this thread:

A general system error occurred: Timed out waiting for vpxa to start

This particular issue was an extremely uncommon one, and so it is unlikely that you will be experiencing this particular issue, though if you are please follow the steps listed in the correct answer.

Finally I would suggest creating a new thread and outlining all of the steps you have taken so far, posting the vpxa.log and output from the various troubleshooting steps in order to cut down on repeated known working steps.

Also the following search should help you out.

https://communities.vmware.com/search.jspa?q=A%20general%20system%20error%20occurred%3A%20Timed%20ou...

Reply
0 Kudos
ManivelR
Hot Shot
Hot Shot
Jump to solution

@ 111zzz888

Thanks for the inputs.It resolved the issue and see the below information for reference.

Error in vCenter while connecting the ESXi host:- A general system error has occurred: Timed out waiting for vpxa to start".

ESXi host error:- VPXA service got stopped after 19 seconds(whenever we restart the VPXA service).There was no VPXA log recorded in VPXA.log.All were the year 2018 logs only.

In addition,we found this error in cat /var/log/syslog.log  error  "/etc/vmware/vpxa' exited after 19 seconds (quick failure 1) 134"

Fix:- We removed the / from <serverIP> & <serverPort> in the beginning and taken vpxa start.It resolved the issue without ESXi reboot.

VPXA.cfg <IoMax>9</IoMax>

      <TaskMax>4</TaskMax>

      <ThreadStackSizeKb>512</ThreadStackSizeKb>

      <threadNamePrefix>vpxa</threadNamePrefix>

    </threadPool>  </vmacore>

   <vpxa>

   <bundleVersion>1000000</bundleVersion>

    <datastorePrincipal>root</datastorePrincipal>

    <hostIp>ESXi IP address</hostIp>

    <hostKey>52f0fc0c-3058-bc0d-4338-40dd3a1abea2</hostKey>

    <hostPort>443</hostPort>

    <licenseExpiryNotificationThreshold>15</licenseExpiryNotificationThreshold>

    <memoryCheckerTimeInSecs>30</memoryCheckerTimeInSecs>

    <serverIp>VC private IP address</serverIp>

    <serverPort>902</serverPort>

  </vpxa> 

Note:- Previoulsy it was  <serverIP/> & <serverPort>

Now the ESXi server got integrated in to vCenter server and vpxa service is stable.

Thank you,

Manivel RR

Reply
0 Kudos