VMware Communities
powermanasdf
Contributor
Contributor

VMware Workstation reboot host when using VT-x

This issue happens first time when I try to start 64-bit guest on 32-bit host, so I initially think it's related to 32/64-bit. But later it become clear it's more about VT-x than about 32/64-bit.

My host OS is 32-bit Hardened Gentoo Linux, kernel 2.6.28-hardened-r9. I've tried both VMware Workstation 6.5.3.185404 and 7.0.1.227600. My hardware is Core 2 Duo 6600 on ASUS P5B-Deluxe with latest stable BIOS 1101. I've power-cycled system, then enabled Vanderpool in BIOS. My CPU doesn't support Trusted Execution Technology, and there no way to disable it in BIOS. I've rebooted several times after that, sometimes with power-cycled, and ensure Vandertool is enabled in BIOS.

I've run VMware-guest64check-5.5.0-18463 tool, and it report "This host is capable of running a 64-bit guest operating system under this VMware product.". I've boot using vt.iso from http://communities.vmware.com/docs/DOC-8978 and here is it report:

CPU 0: VT is enabled on this core

CPU 1: VT is enabled on this core

When I start any (32/64-bit) guest in "Intel VT-x or AMD-V" mode, my host OS immediately reboot (right after guest "Power On", I don't even see guest's BIOS POST output). When I start 64-bit guest in "Intel VT-x/EPT or AMD-V/RVI" I see guest BIOS POST, then guest OS start booting (Windows7 install dvd completes progressbar "Loading files"), and then host OS reboot. And "reboot" actually mean skipping normal host OS shutdown procedure and immediately reset system to host BIOS POST.

I've tried to boot 32-bit guests (Windows7, Ubuntu9.04 and Gentoo) using all possible virtualization modes. In "Automatic", "Automatic with Replay", "Binary translation" everything works, in "Intel VT-x/EPT or AMD-V/RVI" I got message "This host does not support EPT. Using software virtualization with a software MMU." and everything works ok too (only 64-bit guests reboot host when this mode used).

About a year ago I tried to disable hardened in kernel to ensure this isn't because of PaX/GrSecurity, but that doesn't help. ASUS provide "beta" BIOS updates, but according to their descriptions these updates doesn't fix this issue, so I'm not sure is it good idea to try it.

My best guess now it's motherboard/BIOS bug. Any ideas?

Tags (2)
Reply
0 Kudos
20 Replies
admin
Immortal
Immortal

Which version of Workstation are you running?

Reply
0 Kudos
admin
Immortal
Immortal

Never mind. I see it above.

Do you have kvm support compiled into your host kernel?

Reply
0 Kudos
powermanasdf
Contributor
Contributor

# gzip -dc /proc/config.gz | grep -i kvm
CONFIG_HAVE_KVM=y
# CONFIG_KVM is not set
Reply
0 Kudos
admin
Immortal
Immortal

Can you post a vmware.log file from the Workstation 7.0.1 build?

Reply
0 Kudos
powermanasdf
Contributor
Contributor

This one is from trying to boot Win7 x64 install dvd in "Intel VT-x/EPT or AMD-V/RVI" mode, i.e. one which reset host OS after finishing text-mode "Loading files" progressbar.

I've tried to attach it several times, but got error:

Internal Server Error - Write

The server encountered an internal error or misconfiguration and was unable to complete your request.

Reference #4.47891645.1270497753.18ca2f

So, I've uploaded it to pastebin: http://pastebin.com/raw.php?i=jh2At6GY

Reply
0 Kudos
powermanasdf
Contributor
Contributor

And this one is for trying to start 32-bit guest Ubuntu 9.04 using "Intel VT-x or AMD-V" - i.e. which one reset host OS right after clicking on "Power On": http://pastebin.com/raw.php?i=5dg49h7H

Reply
0 Kudos
admin
Immortal
Immortal

Hmmm. I do see evidence of a bug (or class of bugs) in Workstation exercised by your CPU, but nothing in your log files would suggest that this class of bugs would result in your host resetting.

This is a long shot, but try shutting down Workstation and manually adding the following options to your configuration file with a text editor:

monitor_control.disable_vpid = TRUE
monitor_control.disable_flexpriority = TRUE

Reply
0 Kudos
powermanasdf
Contributor
Contributor

No way. I've added these lines to 32-bit Ubunti vmx file, then opened/closed/opened vmware to make sure these lines wasn't removed from config by vmware, then try to "Power On" in VT-x mode - host OS reset, as usually.

Reply
0 Kudos
admin
Immortal
Immortal

Can you use VT-x successfully with other virtualization software (e.g. VirtualBox)?

Reply
0 Kudos
powermanasdf
Contributor
Contributor

No, I never tried any other virtualization software.

Reply
0 Kudos
powermanasdf
Contributor
Contributor

I've tried VirtualBox, but it doesn't work on my system. I don't think issue with VirtualBox is related to this one, but just in case details are here: http://bugs.gentoo.org/show_bug.cgi?id=313607

Any other ideas what to try next?

Reply
0 Kudos
golddiggie
Champion
Champion

Any possibility you could move from gentoo to CentOS 5.4 ()? If you have more than 4GB of memory inside the host, get the 64 bit version...

VMware VCP4

Consider awarding points for "helpful" and/or "correct" answers.

Reply
0 Kudos
admin
Immortal
Immortal

At this point, I'd try replacing the CPU.

Reply
0 Kudos
powermanasdf
Contributor
Contributor

Why do you think so?!

Issues with VirtualBox doesn't look similar to issues with VMware at all, and doesn't look related to CPU at all.

Also, this system work few years without any hardware issues - Gentoo, WinXP and WinSeven works ok, Gentoo updates every few days, so a lot of compilations and a lot of different software versions works ok these years, VMware works ok without VT-x too.

So, for me it looks like two completely different unrelated bugs. And only host reset by VMware with VT-x may have something with buggy hardware (unlikely, but this may be even lack of power supply, if CPU try to eat at some moment a lot of power when VMware activate VT-x - lack of power explain host OS reset behavior).

Reply
0 Kudos
admin
Immortal
Immortal

Issues with VirtualBox doesn't look similar to issues with VMware at all, and doesn't look related to CPU at all.

I disagree. There are times during the switch between the host OS and the VMware hypervisor when we run without a #GP entry in the IDT. A general protection fault would then lead to a triple fault, which would reset your processor. Since you are experiencing unexpected #GPs with VirtualBox, I suspect that the same unexpected #GPs are the source of your host resets under VMware.

Also, this system work few years without any hardware issues - Gentoo, WinXP and WinSeven works ok, Gentoo updates every few days, so a lot of compilations and a lot of different software versions works ok these years, VMware works ok without VT-x too.

I'm suggesting that perhaps it is something in the VT-x circuitry which is faulty. The rest of the CPU is probably fine. Of course, this is only a conjecture.

Reply
0 Kudos
ndebaggis
Contributor
Contributor

I'm seeing the same issues. This is related to custom grsecurity/pax kernel builds... first noticed running a guest inside WS 7 on Fedora 12 x86_64 host laptop. The laptop reboots as soon as the guest starts to load. It'd be nice to have a solution to this issue without having to compromise the grsecurity/pax settings...This log snippet was caught running the above config inside ESXi host, so: (ESXi 4.0U1 (FC12 x86_64 grsecurity kernel WS7 (any guest OS))) where FC12 is the virtual machine referenced below...

May 04 00:13:33.626: vcpu-0| Triple fault.

May 04 00:13:33.626: vcpu-0| Msg_Hint: msg.monitorEvent.cpl0SS (sent)

May 04 00:13:33.626: vcpu-0| *** Virtual machine kernel stack fault (hardware reset) ***

May 04 00:13:33.626: vcpu-0| The virtual machine just suffered a stack fault in kernel mode. On a real computer, this would amount to a reset of the processor. It can be caused by an incorrect configuration of the virtual machine, a bug in the operating system, or a problem in the VMware ESX software. Press OK to reboot virtual machine or Cancel to shut it down.

May 04 00:13:33.626: vcpu-0| -


May 04 00:13:33.629: vcpu-0| CPU reset: hard

Reply
0 Kudos
admin
Immortal
Immortal

I'm not sure that these two incidents are the same, since the second one has nothing to do with VT-x.

From the VM you provided, we have been able to determine that, as a result of the grsecurity patches, a dynamically allocated code page used by VMware Workstation is marked "no-execute." It is the attempt to execute code on that page that results in a host reboot. I've filed a bug report on this issue.

Reply
0 Kudos
powermanasdf
Contributor
Contributor

According to http://vip.asus.com/forum/view.aspx?board_id=1&model=P5B+Deluxe&id=20090612063511362&page=1&SLanguag... my motherboard doesn't support VT-D, only VT supported. Can this be reason for my issue?

Reply
0 Kudos
admin
Immortal
Immortal

No. Workstation does not use VT-d.

The problem is a bug in the grsecurity patches. They've broken vmap in such a way that vmap(..., VM_PAGE_KERNEL_EXEC) may map a page as non-executable, despite the flag requesting an executable mapping. We've reported the bug to them.

Reply
0 Kudos