All VMs random shutdown

[Smily1]

Every couple of weeks all of my VMs just power down!!! This has happend 4 times now.

The ESXi host keeps running fine.

I have since restricted all resources on each VM to less than 50% of resources I.E. 1 core and only 2GB ram. VMs still power down after 2 weeks.

there are only 2 VMs.

FreeBSD 7.2

Server 2008 x64

Thanks!

Nathan

System Specs.

Dell PowerEdge 1950

2x 3.0Ghz dual core Woodcrest Intel CPU

4x 2GB FBDIMM

PERC 5i RAID controller

All latest BIOS/Firmwares as of 2 months ago.

[Smily1]

Local logging setup.

Now to wait for a reboot.

Thanks guys,

Nathan

[Smily1]

Yea could definitely be hardware. I did have it running for almost a month locally before moving it to the datacenter.... No problems then...

-Nathan

[DSTAVERT]

I agree with Troy. Hardware starts to look suspicious. I don't have Dell equipment so I don't know what information you can get back from DRAC. Does OpenManage connect to DRAC? Getting viewable realtime feedback could be useful. Memtest wouldn't hurt.

[Dave_Mishchenko]

When you look at the log file that was generated from the vSphere client you'll want to open the tgz that is created and the look at /var/log/message (and messsages.X.gz) as well as /var/log/vmware/vmware/hostd.log (and hostd.X.gz). If you have a specific time that a VM shutdown at, then look at that time period (the logs will be in UTC).




Dave

VMware Communities User Moderator

New book in town - vSphere Quick Start Guide -http://www.yellow-bricks.com/2009/08/12/new-book-in-town-vsphere-quick-start-guide/.

Do you have a system or PCI card working with VMDirectPath? Submit your specs to the Unofficial VMDirectPath HCL - http://www.vm-help.com/forum/viewforum.php?f=21.

[AntonVZhbankov]

This is the only client log file. Include ESXi when exporting logs.

---

MCSA, MCTS, VCP, VMware vExpert '2009

http://blog.vadmin.ru

[Smily1]

Doing File>Export>Export system logs...

does not export any .gz files. The only relevant log that is exported is viclient-5.log

-Nathan

[DSTAVERT]

I don't think you will find anything prior to the shutdown. If you moved log destination are they showing up in your new location?

[Smily1]

yes...

Logs are showing up in new location. I already downloaded one through vSphere.

-Nathan

[DSTAVERT]

I shouldn't have dismissed Dave's suggestion to check. Sorry and apologies Dave.You really should check to see if anything did survive. Use the unsupported console and have a look in /var/log and /var/log/vmware

[DSTAVERT]

No matter what do check back and let everyone know what happens.

Good luck.

[Smily1]

Will Do!

Thanks for all of your help!

-Nathan

[Dave_Mishchenko]

Your comment is correct - if you've rebooted then messages and hostd.log will be gone. My comment was more that the export of system logs should have produced more files that just the client file.

Nathan, when I export the system logs a folder is created. With that folder I get a file like 192.168.1.56-vmsupport-2009-10-23@13-30-57.tgz - that'll contain the important files. Can you also post one of the vmware*.log files for a VM that shutdown?




Dave

VMware Communities User Moderator

New book in town - vSphere Quick Start Guide -http://www.yellow-bricks.com/2009/08/12/new-book-in-town-vsphere-quick-start-guide/.

Do you have a system or PCI card working with VMDirectPath? Submit your specs to the Unofficial VMDirectPath HCL - http://www.vm-help.com/forum/viewforum.php?f=21.

[winsrock]

Check the VMware Alerts Setting, similar problem i faced when i did changes in VMware alarm action, i selected VM Images Off if CPU/Memory usage is above 75%,

but my intention were to get the alert emails if any VM images are getting down or up, somehow i realised my images are getting down due to Alarm Action and i removed all the options.

Hope this will help you to solve the issue.

Jaweed.

[Tanav]

please upgrade the Perc Controller drivers and see the result. we found that drive not upgraded lead to lossing connectivity with local storage.

[Nikhil_Patwa]

Hi,

Did u check the VM permissions maybe you have set the VMs to reboot/shutdown every after 2 weeks.

I don't think there is anything to do with the VM since both are different VMs - 1 Microsoft and 1 Linux, can't be a bug.

Regards,

Nikhil

[Tanav · Accepted solution]

how to check the VM permissions. please explain

[J1mbo]

Is this server managed through Dell OpenManage? Possibly the machine is hanging causing an ASR event - could be degraded CPU or RAM perhaps.

Please award points to any useful answer.

[Smily1]

@ Tanav

PERC drivers? in ESXi ? or are you referring to the Firmware on the controller?

[Smily1]

@Nikhil Patwa

No they are not scheduled to reboot every 2 weeks... Not unless thats a VMware ESXi default.

They are randomly rebooting.... But it usually takes about 2 weeks between reboots.

[Smily1]

Dang it accidentally clicked the "Answered" button.... Man no "are you sure" prompt!

How Do I change it back to Un Answered...???

-Nathan