VMware

This Question is Possibly Answered

1 "correct" answer available (10 pts) 2 "helpful" answers available (6 pts)
13 Replies Last post: Feb 14, 2007 8:11 AM by bob.bogo  

Slow guest interactive response times posted: Dec 27, 2005 5:46 PM

Click to view s.c.v's profile Lurker
s.c.v
5 posts since
May 2, 2005
Setup:
VMware: 5.5.1 (build 19175)
Guest OS: WinXP-SP2
Host OS: SuSE-10.0 (kernel 2.6.13-15.7)
Host H/W: Dell M70 laptop (2.13GHz Pentium-M with 2GB RAM)

Symptom: Guest OS has very sluggish interactive response times. For example, it takes several seconds to bring up a small pop-up menu on the root window if the "Fade" effect is enabled (yes I know you can turn this off, but it is a quick way to measure of what is wrong here). Logouts take several minutes. The guest clock is very slow if the host-sync setting is disabled in the VMware tools dialog.

Work-around: By loading down the host OS a bit (e.g., using a background "ls -Rl /" command), the guest OS response becomes much better. This is a stupid work-around, but the best I can come up with so far.

My best guess is that this is a problem with ACPI and/or SpeedStep? I have tried the following so far with no significant benefit:

1) Per VMware FAQ "Answer-ID" 892 ("Virtual Machine Seems Slow when Running a..."), I've experimented with the "host.useFastclock" setting in my .vmx file. and with building a new host kernel (2.6.14.5) with a modified "#define HZ 1000" (per VMware FAQ "Answer ID 892).
2) Per VMware FAQ "Answer-ID" 708 ("Locking SpeedStep on Host to Constant Speed") I've set the host ACPI settings to the "Performance" settings.

Any suggestions?

Re: Slow guest interactive response times

1. Dec 27, 2005 6:57 PM in response to: s.c.v
Click to view whateley's profile Expert
whateley
533 posts since
Nov 29, 2004
Your diagnosis sounds correct, the host is slowing the CPU speed when the guest needs it fast.

Do you have any cpu speed control set up in the host OS? And if so, can you disable it? You may want to try a utility like CPUspeed.

Re: Slow guest interactive response times

2. Dec 27, 2005 8:20 PM in response to: whateley
Click to view s.c.v's profile Lurker
s.c.v
5 posts since
May 2, 2005
I'm definitely running the "performance" frequency scaling governer and I've modified the scaling_min_freq value to be the same as the max value. I loaded the "cpufreq_stats" module and confirm that the cpu is running at full speed with no state transitions:
--------------------
cat stats/time_in_state
2133000 77413
1867000 0
1600000 0
1333000 0
1067000 0
800000 0
------------------

Further observation: If I open the "Date & Time Properties" dialog box on my XP Guest OS (right click on the task bar and select "Adjust Date/Time", this displays a nice clock face with a second hand. The second hand only moves forward one second occasionally (once every 15 seconds or so) when the host os is relatively idle. When I load down the host OS with some command (e.g., "ls -Rl /") the second hand moves forward normally.

Re: Slow guest interactive response times

3. Dec 28, 2005 8:38 AM in response to: s.c.v
Click to view whateley's profile Expert
whateley
533 posts since
Nov 29, 2004
What is the nice value of the vmware processes? Try bumping them down a bit.

Re: Slow guest interactive response times

4. Dec 28, 2005 12:30 PM in response to: whateley
Click to view s.c.v's profile Lurker
s.c.v
5 posts since
May 2, 2005
The vmware-vmx process contains 7 threads, all running at nice=(-10). I changed them down to (-15) with no effect. Thanks for the suggestion though.

This does still seem to be some sort of problem with the host scheduler and/or host/guest timer interaction since the virtual machine is apparently being cpu-starved.

I also experimented with the IOSCHED settings using the "ionice" command. No significant improvement with any of the options there.

Re: Slow guest interactive response times

5. Dec 28, 2005 1:36 PM in response to: s.c.v
Click to view whateley's profile Expert
whateley
533 posts since
Nov 29, 2004
Time for the big guns... :) Have you had a look at the VM's *.log file? You can also try to add debug = "TRUE" to the VM's vmx file. Perhaps more info is hiding in the logs.

Re: Slow guest interactive response times

6. Jan 5, 2006 12:41 AM in response to: whateley
Click to view jrj's profile Novice
jrj
12 posts since
Nov 25, 2005
I have this problem as well using suse linux 10, and I've found two ways to work around it.

1. set noacpi as a host kernel boot option
2. set the scaling govenor to performance and put a load on the host machine. I run a do nothing shell script at nice 19 to provide a load to the machine while running vmware, since turning off acpi breaks to many things.
nice 19 busyloop
cat bin/busyloop
while ( true ) ; do a=$a; done

As I understand the problem (and I could be wrong) there is a problem with the idle task using C3 sleep states, hence putting a load on the system keeps linux from scheduling the idle task and vmware runs much better (well at least for me).

Re: Slow guest interactive response times

7. Apr 25, 2006 5:21 PM in response to: jrj
Click to view dplatt's profile Novice
dplatt
6 posts since
Apr 16, 2006
Thanks very much for your suggestion as to forcing a "nice busy" condition. Reading it has just gotten me "un-stuck".

I've run into the same problem with horrible slowdowns using a Windows 98 guest on an Asus laptop with 2.6.15.7 kernel. The Windows clock would simply stop incrementing at all, and Windows wouldn't run worth a darned, unless I forced some sort of activity by (e.g.) mousing down on the Start taskbar and waving the cursor around the windows.

Doing a highly-nice busy loop in another Linux window seems to allow the Windows VM to run pretty successfully. I don't know if it's keeping perfect time, but it's close enough to real-time (and responsive enough) to make the Windows guest usable. Interestingly, although the CPU is pegged, SpeedStep is still holding the CPU at its lowest speed... I'm not sure whether that's because I haven't used the keyboard/mouse in a while, or whether there's specific code to cause heavily-niced processes to prefer the lower CPU state.

It certainly seems as if there's a problem in the VmWare vmmon, having to do with scheduling. It's as if vmmon (or whatever process/thread is handing the individual virtual machine) isn't setting itself up in a run-state which would cause the Linux scheduler to run it at appropriate times... e.g. it's not using a timer or other explicit wakeup. Since it isn't asking for an explicit wakeup, the Linux ACPI code and the BIOS ACPI end up putting the system into a sleep state, and the virtual machine isn't run until something else happens to awaken the system from its short-term sleep.

Sounds as if the VmWare kernel team needs to take another look at how the 2.6 kernel, ACPI, and vmmon interact!

Re: Slow guest interactive response times

8. Apr 25, 2006 9:32 PM in response to: jrj
Click to view dplatt's profile Novice
dplatt
6 posts since
Apr 16, 2006
I think I found a more elegant workaround.

The delay to the virtual machine seems to occur if the CPU goes idle, and ACPI promotes the idle CPU from ACPI state C1 to a higher state. The documentation say that in the higher states, the CPU takes longer to wake up out of its sleep... and I suspect that on some systems the C2 state may be preventing the firing of certain interrupts which would wake up the system and allow the virtual machine to run and keep proper time.

The workaround:

cat /sys/module/processor/parameters/max_cstate
echo 1 > /sys/module/processor/parameters/max_cstate

This will prevent ACPI from promoting from state C1 to C2. It seems to let the virtual
machine keep up with real-time pretty well. The guest-machine time still drifts behind
real-time when the guest and host are both mostly idle, but the guest catches up once it starts running an activity of some sort.

To undo the workaround, echo the original value back into max_cstate

Re: Slow guest interactive response times

9. Jul 28, 2006 10:26 AM in response to: s.c.v
Click to view danslay's profile Novice
danslay
9 posts since
Apr 7, 2005
Has this been resolved yet?

I'm having the same problem, but none of the above solutions has helped, although I did not try disabling ACPI. (running ls -Rl / did nothing, nor did nice'ing things, nor did changing the max_cstate - I did restart all vmware services after making that change).

I am running on a 64-bit Fedora Core 5 host. Odd thing is, have an almost identical machine that does not have this problem. The problem machine has the same OS, same mobo, slightly newer/faster Opteron CPUs, same Windows 2000 Pro guest OS, but different disk setup (one has SCSI system disk, other has SATA).

The vmx files look similar, and where they arent, changing the problem machines vmx to match the other did not affect this; the problem machine vmx file did not have this setting: checkpoint.vmState = "". But the problem machine did have this setting, which was not on the working machine: MemAllowAutoScaleDown = "FALSE". So I removed the MemAllowAutoScaleDown, and added the checkpoint one, but no change in performance.

Any ideas??

Resolved for 64-bit system (Tyan 2885 / dual opteron)

10. Aug 4, 2006 5:25 AM in response to: danslay
Click to view danslay's profile Novice
danslay
9 posts since
Apr 7, 2005
Resolution update:

On the system having problems here, it turns out a new BIOS update was responsible. The new BIOS (AMIBIOS, for a Tyan 2885 dual opteron board) has a setting for Burst Length = 8 beats (default after bios flash update). (This is found under under Chipset --> NorthBridge --> Memory). Setting this to 4 beats (the only other option) has completely solved the problem. Within the bios, it says for this item: "64 bit Dq must use the 4 beats", which led me to try changing it, since this is a 64-bit system (although I have no idea what a Dq is).

Re: Slow guest interactive response times

11. Jan 17, 2007 10:46 AM in response to: dplatt
Click to view bob.bogo's profile Lurker
bob.bogo
5 posts since
Nov 9, 2006
This works perfectly for me on my x86_64 Fedora Core 6 laptop. The 4x-too-slow clock was really begining to bug me. Cheers!

(see http://www.vmware.com/community/message.jspa?messageID=553624#553624 for details)

Re: Slow guest interactive response times

12. Jan 27, 2007 7:40 PM in response to: bob.bogo
Click to view jnevik's profile Lurker
jnevik
5 posts since
Jan 27, 2007
How did you install on Fedora Core 6? I get a compile error when running vmware-config.pl

Do you accept? (yes/no) yes

Thank you.

Configuring fallback GTK+ 2.4 libraries.

In which directory do you want to install the mime type icons?
[/usr/share/icons]

What directory contains your desktop menu entry files? These files have a
.desktop file extension. [/usr/share/applications]

In which directory do you want to install the application's icon?
[/usr/share/pixmaps]

Trying to find a suitable vmmon module for your running kernel.

None of the pre-built vmmon modules for VMware Server is suitable for your
running kernel. Do you want this program to try to build the vmmon module for
your system (you need to have a C compiler installed on your system)? [yes]

Using compiler "/usr/bin/gcc". Use environment variable CC to override.

What is the location of the directory of C header files that match your running
kernel? [/lib/modules/2.6.18-1.2798.fc6/build/include]

Extracting the sources of the vmmon module.

Building the vmmon module.

Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-config0/vmmon-only'
make -C /lib/modules/2.6.18-1.2798.fc6/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. modules
make[1]: Entering directory `/usr/src/kernels/2.6.18-1.2798.fc6-x86_64'
CC [M] /tmp/vmware-config0/vmmon-only/linux/driver.o
CC [M] /tmp/vmware-config0/vmmon-only/linux/hostif.o
CC [M] /tmp/vmware-config0/vmmon-only/common/cpuid.o
CC [M] /tmp/vmware-config0/vmmon-only/common/hash.o
CC [M] /tmp/vmware-config0/vmmon-only/common/memtrack.o
CC [M] /tmp/vmware-config0/vmmon-only/common/phystrack.o
CC [M] /tmp/vmware-config0/vmmon-only/common/task.o
CC [M] /tmp/vmware-config0/vmmon-only/common/vmx86.o
CC [M] /tmp/vmware-config0/vmmon-only/vmcore/moduleloop.o
LD [M] /tmp/vmware-config0/vmmon-only/vmmon.o
Building modules, stage 2.
MODPOST
CC /tmp/vmware-config0/vmmon-only/vmmon.mod.o
LD [M] /tmp/vmware-config0/vmmon-only/vmmon.ko
make[1]: Leaving directory `/usr/src/kernels/2.6.18-1.2798.fc6-x86_64'
cp -f vmmon.ko ./../vmmon.o
make: Leaving directory `/tmp/vmware-config0/vmmon-only'
The module loads perfectly in the running kernel.

Do you want networking for your virtual machines? (yes/no/help) [yes]

Configuring a bridged network for vmnet0.

Your computer has multiple ethernet network interfaces available: eth0, eth1.
Which one do you want to bridge to vmnet0? [eth0] eth1

The following bridged networks have been defined:

. vmnet0 is bridged to eth1

Do you wish to configure another bridged network? (yes/no) [no] yes

Configuring a bridged network for vmnet2.

The following bridged networks have been defined:

. vmnet0 is bridged to eth1
. vmnet2 is bridged to eth0

All your ethernet interfaces are already bridged.

Do you want to be able to use NAT networking in your virtual machines? (yes/no)
[yes]

Configuring a NAT network for vmnet8.

Do you want this program to probe for an unused private subnet? (yes/no/help)
[yes]

Probing for an unused private subnet (this can take some time)...

The subnet 192.168.67.0/255.255.255.0 appears to be unused.

The following NAT networks have been defined:

. vmnet8 is a NAT network on private subnet 192.168.67.0.

Do you wish to configure another NAT network? (yes/no) [no]

Do you want to be able to use host-only networking in your virtual machines?
[yes]

Configuring a host-only network for vmnet1.

Do you want this program to probe for an unused private subnet? (yes/no/help)
[yes]

Probing for an unused private subnet (this can take some time)...

The subnet 172.16.81.0/255.255.255.0 appears to be unused.

The following host-only networks have been defined:

. vmnet1 is a host-only network on private subnet 172.16.81.0.

Do you wish to configure another host-only network? (yes/no) [no]

Extracting the sources of the vmnet module.

Building the vmnet module.

Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-config0/vmnet-only'
make -C /lib/modules/2.6.18-1.2798.fc6/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. modules
make[1]: Entering directory `/usr/src/kernels/2.6.18-1.2798.fc6-x86_64'
CC [M] /tmp/vmware-config0/vmnet-only/driver.o
CC [M] /tmp/vmware-config0/vmnet-only/hub.o
CC [M] /tmp/vmware-config0/vmnet-only/userif.o
CC [M] /tmp/vmware-config0/vmnet-only/netif.o
CC [M] /tmp/vmware-config0/vmnet-only/bridge.o
CC [M] /tmp/vmware-config0/vmnet-only/procfs.o
/tmp/vmware-config0/vmnet-only/procfs.c:33:26: error: linux/config.h: No such file or directory
make[2]: *** [/tmp/vmware-config0/vmnet-only/procfs.o] Error 1
make[1]: *** [_module_/tmp/vmware-config0/vmnet-only] Error 2
make[1]: Leaving directory `/usr/src/kernels/2.6.18-1.2798.fc6-x86_64'
make: *** [vmnet.ko] Error 2
make: Leaving directory `/tmp/vmware-config0/vmnet-only'
Unable to build the vmnet module.

For more information on how to troubleshoot module-related problems, please
visit our Web site at "http://www.vmware.com/download/modules/modules.html" and
"http://www.vmware.com/support/reference/linux/prebuilt_modules_linux.html".

Execution aborted.

Re: Slow guest interactive response times

13. Feb 14, 2007 8:11 AM in response to: jnevik
Click to view bob.bogo's profile Lurker
bob.bogo
5 posts since
Nov 9, 2006
Try putting

"linux/config.h: No such file or directory" vmware

into google. (You may find something like this thread: http://www.vmware.com/community/message.jspa?messageID=501043#501043.) It just seems to work for me (VMwareWorkstation-5.5.3-34685).

Actions

VMware Developer

SDKs, APIs, Videos, Learn and much more in the Developer community.

Learn More

Developer Sample Code

Increase your developer productivity with VMware API sample code.

Learn More

VMworld Sessions & Labs

Online access to the latest VMworld Sessions & Labs and online services.

Learn more

Purchase PSO Credits Online

Purchase credits to redeem training and consulting services online.

Buy Now

Community Hardware Software

View reported configurations or report your own.

Learn More

VMware vSphere

Come witness the next giant leap in virtualization.

Register Today

Communities