FreeBSD 8.0 64 bit causes hard freeze of ESXi server

[outzider]

I have a brand new installation of a FreeBSD 8.0 host in 64 bit mode. It can run idle for weeks on end, but when I execute 'portupgrade', or on one occasion, simply installing a port (which is a compile), the entire ESXi server suffers a hard freeze. I am unable to pull up any events from the process. When plugging in a video cable to the machine, the display is garbled -- the idle screen is moved in chunks to the corner of the screen. It has normal characters, but it almost looks like a failed screen refresh. The machine becomes unresponsive to any network activity or console keypresses.

I recently updated to Update 1, and the exact same problem occurs.

This is duplicated on a FreeBSD 7.3 installation in 64 bit. I have not seen a FreeBSD 32 bit installation do the same thing, and the Ubuntu Linux 64 bit installation has not done this either. Hardware successfully completes PC-Doctor hardware diagnostics, including CPU and RAM tests. The console reports no errors in any hardware.

Has anyone run into anything similar in FreeBSD? If so, were you able to resolve? Do I need to move to a 32 bit installation of FreeBSD, or attempt to migrate my stuff to Linux?

Machine:

AMD Athlon II 620 Quad Core processor

MSI K9N6PGAM2-V2 motherboard

Dell PERC 5 RAID controller

4x Seagate 1TB SATA2 Hard Drives connected as one logical drive through the PERC5

[outzider]

I've just duplicated this issue under FreeBSD 8.0 32 bit.

[DSTAVERT]

Not that this is an answer but FreeBSD 8 doesn't appear as a supported guest on the Hardware Compatibility List .

[outzider]

Yeah, I just noticed that myself, despite their consumer offerings having decent support, as well as tools being included. This is unfortunate.

[w6vms]

I have five FreeBSD 8.0 VM's running without any problems on ESXi 4.0U1. They all started as 8.0RC1, and all got upgraded to 8.0-RELEASE over the last few days, so lots of I/O and CPU intensive tasks. In addition, I have installing many ports on all of them.

So far, I have not seen the behavior that you report. The host is a SuperMicro motherboard with a single CPU (3.0 GHz Pentium D) with 4GB RAM and a single local SATA drive.

-jav

[outzider]

That gives me the tiniest hint. I have to run into the colo and reboot the machine, but maybe this has something to do with the AMD-V virtualization extensions. As far as I know, the Pentium D didn't have VT-X, and maybe that's causing this issue.

[w6vms]

Have you tried FreeBSD 7.2? I am asking because I do have a FreeBSD 7.2-RELEASE VM running on a host with Xeon processors, it has been very stable for about a year now.

-jav

[outzider]

I remember trying 7.2 previously and getting weird ssh disconnect issues, but I may have to try it again. I have a pfSense VM that is running off of a custom 7.2-RELEASE install, and it hasn't affected anything, though nothing involving a lot of disk I/O or installations happen in that VM.

[Risner]

This is likely an AMD problem. I've never been able to reliably run ESXi on AMD. I tried on a Tyan S2882-D and on a Arima HDAMA (both using Opteron 200 series cpu's.) Both had hard lockups, purple screens of death, etc.

I switch to using Core i7 and/or Core2 Quad chips and I've never had any issues. I run mostly FreeBSD from 6.x boxes to 7.x to 8.0RC1-3 and most have been updated to 8.0-stable.

No crashes.

[Scissor]

Hard drive going bad, maybe? When you reboot your system, check your PERC raid controller screens to see if it is reporting any disk problems.

[outzider]

All the drives were new when put in here, and went through a sector scan before the installation occurred. The PERC controller shows nothing wrong with any of the four drives, and even so, one failed drive in a RAID 10 shouldn't cause a hard crash.

[w6vms]

Did you ever figure this one out?

-jav

[outzider]

Unfortunately, no. I'm pretty much willing to accept the 'AMD isn't great in ESXi' idea at this point, as I had no problems with an earlier Core 2 Duo install. I attempted moving all of the VMs to 32 bit, but managed to duplicate the issue. I changed settings to disable the AMD virtualization extensions, and still saw the same issues. Moving to FreeBSD 7 seemed to make it last slightly longer, but under heavy IO, it still brought the machine down. My next machine will probably go Intel and I'll try again, but for now, I'm running on a stripped Linux install and VirtualBox. VMWare Server was just too slow and resource hungry.

So, I'll probably be back in a few months to whine about something else. Thanks for the ideas, though.