Continuing ESX 3.0.2 crash... not hardware related

[REALM]

Over the last 6 months, I've had an ESX box crash into like a pink or purple screen, and it is not hardware related. I have updated firmware, replaced motherboards, memory boards, ran full diagnostics on all, and not the same ESX crashes each time, but there is a pattern.

The last 4 times they've crashed, there are 2 particular virtual machines running on the crashed box. (I don't have records of what VM's were where before the last 4 crashes). Whatever physical ESX box those VM's are on, it's that ESX box that crashes, and I'm at a lost to determine how they would cause it. Is there somewhere I can send event logs or something for those boxes for VMWare to look at?

I have 3 HP DL580 G2's running ESX 3.0.2. Both VM's that seem to cause the crash are Win2003 servers. One is simply a print server, and the other is a web server with a web application built by an outside vendor.

Anyone know what on a Win2003 VM would cause an ESX box to crash completely?

[BenLe]

Did you try re-installing the VMware tools?

[REALM]

I thought VMWare Tools just helped with graphics and mouse performance? Per your suggestion though I have gone ahead and reinstalled VMWare tools to see if it helps.

[REALM]

Is it possible that the entire ESX would crash if say a server wasn't allocated enough CPU/Memory resources?

Say I had a Win2003 server with 768MB of RAM and 1 vCPU, and it shoul dhave been more like 1280MB and 2 vCPU.

Could that cause the entire ESX to crash?

[mattinator]

What is the CPU/Mem utilization for these two machines? Does it show high percentage utilization for the cpu/mem for these two guests? I wouldn't think that adding more memory/cpu would help instead I would think that it would hurt. What do you have your reservations/shares set to for these machines? Do you have limits set for them?

[REALM]

Performance reporting for my entire vmware system is unreliable. If I look at a report from say the last week or month, it will show a nice smooth line at about 20%-50% fluctuation for all my machines. However, if I look at real-time data, often times it is hovering at 60% and spiking to 90%, but the week/monthly reports never show that. So real-time is the only thing I have to look at unfortunately. When I figured that out, looking at real-time reports, yes, these 2 servers were using most of their resources constantly and hitting 100% at a few spikes.

Where do you set the reservation shares?

[hitchhiker]

Is there a backtrace printed on the pink/purple screen when the machine crashes? Can you scribble that down and report it?

In anycase, do take a support dump (vm-support or something like that on the cli) and send it to VMware - this might be a known issue.

..

[mattinator · Accepted solution]

You assign reservations for a guest system in Virtual Center, if you have it, or by editing the properties of the guest under options. It would be helpful to look at the ESX Resource Management Guide which will give you the definitions for reservations and shares and tell you how to use them effectively. I bet this would help with the overall reliability and performance of the server.