Guest CPU Usage High in comparison to HOST.

[JamesSykes]

I've got 2 ESXi 4.1 hosts, identical in every way. 8x 2.67ghz CPUs and 32GB RAM.

On each I have a guest running Windows 2008 R2 with 4GB ram and 1 vCPU.

On each I run Prime95 for 5 minutes....

One Host1 VMware Performance tab reports the guest is using 2600mhz CPU. As expected.

One Host2 VMware Performance tab reports the guest is using 500mhz CPU. Not expected.

This corresponds with the fact that the server on HOST2 is running extremely slowly because it is unable to fully utilise it's processor.

This is typical behaviour when for example a guest is being throttled or is in a resouce pool and not able to fully utilise the processor, in which case taskmanager will show you very high CPU.

I have no resource pools and no limits or anything else setup.

Any ideas?

[rickardnobel]

Do you have vMotion? What will happen if you move the slower server to HOST1 and try the prime95 stress tool?

(And you are sure there is no CPU limit on this VM? It sounds very much like this of course.)

What does the host CPU usage show? There is no other heavy stuff running on host2 that makes less time available for the guest?

[JamesSykes]

No vMotion setup.

Each host is seriously underutilised, showing a maximum of 5% TOTAL cpu usage, so no throttling should be going on there.

I cannot see a difference between the two hosts in terms of configuration.

I've tested multiple guests on each of the hosts, each show the same issue, so the problem seems to be specific to the host and not the guest.

Like I say it's almost as if there is a CPU limit set on the guest... but there isnt

[rickardnobel]

If you look at the Performance charts for the guest, do you see any Ready Time values reported?

[JamesSykes]

Good idea!

I had a look and as you guessed the VM's on this particular host have got very high CPU ready times.

One VM has an average of 6000ms and a max of 12000ms ready time.

The strange thing is, this would be understandable if we were loading up the boxes...we aren't.

This particular host has got 6 VM,s only 2 of those are doing anything and the total CPU usage for the host being reported is less than 5%. The host is only using 1000mhz of CPU and it has got 8x2.6Ghz CPUs available.

So what causes high ready time other than CPU contention?

Also I should add:

HT is turned on, 8 actual cores, 16 logical.

Of the 6 vm's, all used to have a single vCPU and we were having performance issues. 6 VMS with 1vCPU each should never have to wait for a CPU to be scheduled on a host with 8 physical and 16 logical cores, that should literally never ever ever ever happen. So why are the CPUS waiting?