ESX3 - How to determine on which physical CPU a given VM is running?

[v_z]

How to determine on which physical CPU a given VM is running?

[bertdb]

you can use esxtop to do that; I don't have the exact commands handy.

You know this can change every 20 milliseconds (50 times per second), right ? How fast can you blink ?

[HarisB]

bertdb covered it - this changes every 20 miliseconds and therefore you can't really see what you want, unless you assign affinity and force VM to run on a specific CPU.

The other way of looking at it is that if you run CPU killer inside VM driving the CPU to 100% utilization, you should see 25% utilization on each of your 4 physical cores (if you have 4, recalculate for your situation)

[v_z]

Hmmm, this is not what I see on production system. There are two VMs with about 80% CPU utilization on one of our ESX3 servers. 2 logical cores (out of 😎 shows 80% CPU usage, the rest of them - 0%. CPU average for that ESX is ~30%. The load is not distributed evenly between the cores, which means that those two VMs are scheduled to run on a specific core all the time (no CPU affinity is set).

[v_z]

In reality CPU load is not evenly distributed between physical CPUs

[glynnd1 · Accepted solution]

I wouldn't expect to see ESX moving a VM from core to core just to give the physical/logical CPUs an even work out. Mainly because this would involve scheduling several VMs off and on the CPUs.

Scheduling VMs on and off CPUs costs cycles & time and requires the CPU cache to moved as well - all this costs, and therefore lowers the over all efficiency of the box. If I can leave some of my VMs where they are, then I save a few cycles. VMs that are consuming 80% of a core are going to need access to a core pretty frequently, so leave them where they are. Now when contention kicks in and I am short of CPU I will take it off the core, let another VM on, and later when the 80% VM needs core time I'll pop it back, and possible on a different core.

This is one of the reasons, in my thinking, why the more cores in a box the better even if they are at slower clock speeds.

[v_z]

Thans David. I think you are correct. But that brings back my original question: how to determine on what physical CPU a given VM is scheduled to run. This may help in troubleshooting performance problems.

[falsehope]

Hello,

I hope that the little info I have can shed some light on this.

Using esxtop to gain insight into what Worlds (vCPUs, etc..) are running on what physical CPU is doable. I'm using ESX 3.5 on a dual quad core blade (8 pCPUs) NON-Hyperthreading. From the console of the esx host run esxtop with root rights (su -). The default view within esxtop is the CPU view, so we just need to add another field to it. Press the "f" key to add additional fields or remove them. Select the "H" field (CPU Summary Stats). Then to get back to the main esxtop screen, press any other key. Now press the "o" key to select the order of the fields (I used ABCDHEfg). OK, now you can see the Groups (GID) and the number of Worlds (NWLD) within the Groups.

In order to see what Worlds are within Group press "e". You will be asked to select the GID, I selected 18 and 19 as they are two VMs that both have 2 vCPUs. From the attached pics you can see that GID 18 has two vCPUs these vCPUs are running on physical 2 and 1. Also notice that the other Worlds that make up this VM are also running of different pCPUs (5,3,2,1). I also expanded GID 19 to show it's vCPU to pCPU mapping.

I included two screen shots with the esxtop time stamp hightlighted in the upper left corner to show that within 12min, the Worlds moved to different pCPUs. This cluster is slow today and the ESX host still moved them.

This is just my 2 cents...

HTH