CPU Affinity for a virtual machine

[puzzledtux]

I have a 8 CPU host on which I have installed ESX 3.5 U3. I have also created a virtual machine Windows 2003 Server SP2 with 2 CPU allocated to it.

I am trying to set CPU affinity to that virtual machine to use 2 of my physical processors. Now I see under the Advanced CPU tab, that I am able to select all of the available 8 physical CPU's. So I am now a bit confused as to if VMware allows scaling my virtual machine to use all of the 8 CPU's even when my virtual machine is only 2 CPU?

Now if the above is true and I am being allowed to set this virtual machine affinity to all of the 8 CPU's, then why is that maximum limit of CPU for a virtual machine is only 4? A dumb question probably but I am really confused ?:|

[Rubeck]

Under normal operation the vmkernel might "move" intructions originated from the VM to another physical core... Using CPU affinity pins it to physical cores of your choice..

It has nothing to do whith what may be reflected in the VM regarding the number of CPUs.,,,

Be aware of doing this could break vMotion, DRS and HA functions of this VM,,,,,

/Rubeck

[puzzledtux]

So does that mean the virtual machine may be entitled to use all of the physical cores if I enable all of them is I manually set the affinity? This would imply that my 2 cpu virtual machine is capable of using more than what is actually alocated to it. BTW I am aware of the issues of setting this affinity in DRS cluster environment.

Just as my name goes..I was puzzled at the availability of all the CPU cores in the advanced cpu option for my 2 cpu virtual machine. I thought the rest of the cpu cores should have been disabled and I should have a 1-1 mapping against the vcpu and actual cores.

[eddiecwf2]

Up to 4 CPU Cores per VM for the current ESX version 3.5

[puzzledtux]

I understand that, but we were discussing the CPU affinity thing in here

[RParker]

CPU affinity thing in here

I think you are over thinking this. CPU affinity is the same as pinning a process in Windows. Your process use a specific CPU or many CPU. With a VM you assign the number of vCPU you want that machine to have, and then pin THOSE CPU to a specific Core on the ESX host. That's all you are doing.

You can't have a VM with 2 vCPU and just because you SEE ALL the cores available on the host, that doesn't mean you are using them ALL, you can set which CPU are able to be utilized by the VM.

Also, why are you trying to do this in the first place? Let ESX handle threading, it does a fine job. When you pin a CPU and those CPU are busy, your VM will suffer, and since you selected only certain CPU your VM is forced to use ONLY those CPU you have selected in the Advanced settings.

[puzzledtux]

This is just a test environment wherein I am looking at the capabilties provided by the virtual infrastructure. I was just curious to know about the affinity settings and get views from our experts in here.

I may not use this feature in a prod at the moment since everything seems to run fine but I just exploring things since this capability has been provided by VMware inspite ESX being able to handle threads as you mentioned. And since VMware has provided it SHOULD have an use case as well

[Ken_Cline · Accepted solution]

Hi puzzled,

Yes, the user interface is a little confusing. As RParker said, basically what you are doing when you assign CPU affinity is identifying the cores on which the vCPU is eligible to execute on. At no time will the VM be scheduled to run on more cores than it has vCPUs, so it will never have access to more capacity than it should.

CPU affinity is one of those features that has been around since "the beginning" and it's usefulness is really somewhat fuzzy. I have heard of cases where users were running Citrix workloads configured with 2 vCPU and assigned CPU affinity to each VM with the result of improved performance. I question this because I have never seen it myself, and having a decent understanding of how the vmkernel scheduler works, it doesn't make a lot of sense.

I would encourage you (and anyone else reading this) to essentially forget that CPU Affinity is an option. Using it can cause significant performance degradation (if, for example, you pin a VM across two NUMA nodes rather than keeping all vCPUs in the same node) and will break features (VMotion, DRS, and who knows what else in the future).

Ken Cline

VMware vExpert 2009

VMware Communities User Moderator

Blogging at: http://KensVirtualReality.wordpress.com/

[puzzledtux]

Thanks Ken for clarifying that the virtual machine will not access more capacity than it should.

[mcsenerd]

I've heard all the blah blah blah about not doing multi vCPU VMs and not using affinity and the rest...but all I can tell you is my personal experience. We have a VM print server that is heavily used and has over 600 queues on it. This thing is super CPU bound...hardly any memory and the disk I/O thus far hasn't been an issue...but the thing chews through CPU like there is no tomorrow. Unfortunately...I don't have the benefit of having hosts with the latest and greatest procs in them...therefore my fastest hosts in this cluster are 2.33 Ghz Intel E5410s. Well...this thing needs everything I can throw at it...so...in the interest of keeping my phone from ringing off the hook, I have this thing set for manual DRS and I have it set to only allow it to execute on cores 1-7. As far as I know the service console is ALWAYS scheduled on core 0 and this thing is full-bore non-stop until quitting time every day. So keeping it off of core 0 prevents issues on this particular host. But hey you know what they say...YMMV...

[depping]

Although it sounds weird Ken, but when you have 2vCPU's it would make sense to set affinity to three Cores cause all the stuff like KVM needs to be scheduled as well and they will be scheduled on the same set of cores you picked affinity on... So although the GOS of the VM doesn't use more cores the VM in total will.

Duncan

VMware Communities User Moderator

-

---

Blogging:

Twitter:

If you find this information useful, please award points for "correct" or "helpful".