"backtrace for process" errors when shutting down

[mbell98]

I have three BL465c blades in a chassis all loaded with ESX 3.5 Update 3. One of the servers has problems shutting down. Looking at the console, after "Shutting down loopback interface: " it does the "Starting killall". At that point, it seems to just hang but I start seeing error messages every couple of minutes like:

Backtrace for process vsh (27564)

Backtrace for process vpxa (27287)

Backtrace for process minilogd (27384)

Backtrace for process webAccess (2415)

Some of the processes, such as WebAccess, vsh, and vpxa are repeated but with different numbers. I'm assuming these are process ID numbers.

Any ideas as to what might be going on or how to fix it?

[Troy_Clavell]

....and you are sure there are no disk space issues on your esxh host?

From the service console type:

vdf -h

[mbell98]

No, doesn't look like it. The root partition "/" has the highest utilization of any and it's only at 32% (see attached screen shot).

[moorecr]

I could have posted the same exact problem, we also have 465c HP blades, I'm doing updates and noticing that one of our blades is giving the same "backtrace for process ..." errors.

I havent yet cycled through and havent completed the non critical updates, I'll update with more info if I can resolve it.

[mbell98]

I swapped out the hard drives in one of my BL465c blades and installed 3.5 Update 3 from scratch. After doing that, I noticed that the one installed from scratch did not exhibit the backtrace problems when shutting down. I decided to install 3.5 Update 3 from scratch on the other two blades and voila! problem solved. Even though they were all three running Update 3, they were originally installed with Update 2 then upgraded to Update 3. It would appear that something along the way got hosed up, causing the backtrace errors.

In any case, it appears that installing Update 3 from scratch fixes the issue.

[deceit]

I just had this issue today too with a BL 465C, esx 3.5 update 3.

The only thing I am doing differently is that I restarted the entire server 3-4 times in quick succession.

I was trying to get into the bios and missed the key press. I waited for the host to boot fully into ESX then intiated a reboot via Virtual Infrastructure, repeated the process 3-4 times.

[GarethWilson]

we had this problem arise when we went to update 4 for esx 3.5, when the server rebooted after the update it would just hang on different backtrace processes, the problem was occuring on all of our HP Blade 465c G5 & G1 servers, the problem seems to be the HP management agents and the IPMI, we installed the latest vmware agents from HP (v8.20) which are here :

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&prodTypeId=37...

before doing update 4 and the install now goes fine and the server boots backup fine (we did also upgrade the bios's to the latest version while we had scheduled downtime)

[atc]

We are having similar issues with ESX 3.5 Update 3 on a bl460.

The other issue we are having is contained in this thread: http://communities.vmware.com/thread/201551 Waiting on IPMI initialization on ESX 3.5 U3

The case we have open with HP is 3605775001 in case the two are connected. Our server was working fine but we installed more memory and since the reboot my service console has been inaccessable from the network.

Thus far I have updated our HP management agents from 8.1.1 to 8.2 but that did not resolve the issue. I was disapointed to learn from this thread that it doesn't sound like Update 4 for ESX is the answer.