VMware Cloud Community
ericsowa
Contributor
Contributor

ESX 3.5 How vcpu uses physical CPU and Cores

We are a little confused how physical cpu and cpu cores work when we provision vm's. I have done a tone of reading these past few days and I want to make sure we are doing this correct. In one cluster we have 3 HP DL 585G2 quad dual core servers. When we build a windows web server VM the idea here is to assign it two vcpu's since that is what we had when we bult physical servers. Now does each vcpu get bound to a physical processor or does both vcpu's live on a single processor or core. I wonder if we are over provisioning vCPU's to vm's. We have some vm's that are quad processors.

I started to look at this vmware KB artical Scheduler Cell Size on Six-Core Processors. Currently our esx servers are sert to VMkernel.Boot.cpuCellSize=0. Is that a issue?

Reply
0 Kudos
11 Replies
azn2kew
Champion
Champion

This should answer your questions regarding CPU scheduling http://communities.vmware.com/docs/DOC-4960/

If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!!

Regards,

Stefan Nguyen

VMware vExpert 2009

iGeek Systems Inc.

VMware, Citrix, Microsoft Consultant

If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!! Regards, Stefan Nguyen VMware vExpert 2009 iGeek Systems Inc. VMware vExpert, VCP 3 & 4, VSP, VTSP, CCA, CCEA, CCNA, MCSA, EMCSE, EMCISA
Reply
0 Kudos
ericsowa
Contributor
Contributor

I have read that and I still don't understand. We have 50 we server vm's on a quad dual core server. Each with 2vcpu. Is each VM with 2 vcpu's using a single phisical processor or spread accross 2 cpu's?

Reply
0 Kudos
azn2kew
Champion
Champion

Here is a good explanation from Massimo.

If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!!

Regards,

Stefan Nguyen

VMware vExpert 2009

iGeek Systems Inc.

VMware, Citrix, Microsoft Consultant

If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!! Regards, Stefan Nguyen VMware vExpert 2009 iGeek Systems Inc. VMware vExpert, VCP 3 & 4, VSP, VTSP, CCA, CCEA, CCNA, MCSA, EMCSE, EMCISA
Reply
0 Kudos
weinstein5
Immortal
Immortal

The vmkernel does and excellent job of scheduling vCPUs to insure a balnced load across across the physical cpu cores - it looks every 20 mSec to balance the load. VMware's recommends 4-8 vCPUs per core - so with your configuration dual quad cores cpus VMware's recommendation would be 64 at most - If you have 50 dual vCPU VMs on a dual quad core host you will probably run into CPU contention because your are talking 100 vCPUs - but you also have an added problem that can and will affect virtual machine perfomance. It is the way the virtual SMP (dual vCPU or quad vCPU) are scheduled - the vCPUs are scheduled simultaneously. So if one vCPU can not be scheduled all will not be scheduled in other words it is easier to schedule one vCPU than 2 vCPUs or 4 vCPUs - that is why best practice is always to start out with a single vCPU and only add additional vCPUs if needed. I think that you will find if you drop all of your VMs you will see improved perfomance equal to if not better than what you had in the physical world -

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful
Reply
0 Kudos
Ken_Cline
Champion
Champion

Hard co-scheduling went away with the release of ESX 3.0. The vmkernel now uses relaxed co-scheduling, so vSMP VMs may be scheduled to run even if there aren't enough pCPUs available to concurrently schedule all vCPUs.

With that being said - I would strongly recommend to the OP to reconsider the decision to deploy all vSMP VMs. While you may have had two CPUs in the physical world, chances are very, very good that they were not being used effectively. Most Windows workloads work quite well with a single CPU - especially server-based workloads. Monitor the utilization before making your sizing decisions (or start small and add only when needed). The same is true of memory resources...most physical Windows systems have WAY more memory than they are actually using. Consider creating your VMs with 512MB of RAM and adding more as you see that memory is a constraining resource.

A quick word of caution - if you convert your VMs from vSMP to uni-processor, make sure that you change to the uni-processor HAL!

Ken Cline

VMware vExpert 2009

VMware Communities User Moderator

Blogging at: http://KensVirtualReality.wordpress.com/

Ken Cline VMware vExpert 2009 VMware Communities User Moderator Blogging at: http://KensVirtualReality.wordpress.com/
Reply
0 Kudos
savantsingh
Enthusiast
Enthusiast

To answer your main question 1 vCPU allocated to a VM would mean that one Physical core on the host would serve this VM's kernal processing requests.

Now this is not mandatory that a single CPU would be reserved for that VM it could rotate as per VMKernal's queue(or scheduling).

If you allocate 2 vCPU's they would be treated as 2 physical cores on the host.

Hope this helps!!

Thanks

If you found this information useful, please consider awarding points for "Correct" or "Helpful".
Reply
0 Kudos
bbricker
Contributor
Contributor

A quick word of caution - if you convert your VMs from vSMP to uni-processor, make sure that you change to the uni-processor HAL!

Stupid question- but can someone please explain this to me? I've never changed anything in Windows- just shutdown, change vSMP (only ever gone up though), then power on. Windows always recognizes the new virtual procs...

Reply
0 Kudos
Chuck8773
Hot Shot
Hot Shot

When you reboot after adding the additional vCPU, did you have to reboot again? It found new hardware and installed it. This was Windows changing from the uniproc HAL to the SMP HAL. When you reduce it to one vCPU it does not change this for you. Go into device manager and you will see the SMP driver listed. If you reduce it to one vCPU, you need to change this manually.

Charles Killmer, VCP

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful".

Charles Killmer, VCP4 If you found this or other information useful, please consider awarding points for "Correct" or "Helpful".
Reply
0 Kudos
bbricker
Contributor
Contributor

When you reboot after adding the additional vCPU, did you have to reboot again? It found new hardware and installed it. This was Windows changing from the uniproc HAL to the SMP HAL.

Ah yes, I believe it does do that, never really thought about it.

When you reduce it to one vCPU it does not change this for you. Go into device manager and you will see the SMP driver listed. If you reduce it to one vCPU, you need to change this manually.

"change it manually"? As in, remove the SMP driver and reboot, or what?

Thanks again

Reply
0 Kudos
Chuck8773
Hot Shot
Hot Shot

This thread should help.

http://communities.vmware.com/thread/176459

Charles Killmer, VCP

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful".

Charles Killmer, VCP4 If you found this or other information useful, please consider awarding points for "Correct" or "Helpful".
Reply
0 Kudos
Texiwill
Leadership
Leadership

Hello,

Moved to the ESX 3.5 forum.


Best regards,

Edward L. Haletky VMware Communities User Moderator, VMware vExpert 2009, Virtualization Practice Analyst[/url]
Now Available: 'VMware vSphere(TM) and Virtual Infrastructure Security: Securing the Virtual Environment'[/url]
Also available 'VMWare ESX Server in the Enterprise'[/url]
[url=http://www.astroarch.com/wiki/index.php/Blog_Roll]SearchVMware Pro[/url]|Blue Gears[/url]|Top Virtualization Security Links[/url]|Virtualization Security Round Table Podcast[/url]

--
Edward L. Haletky
vExpert XIV: 2009-2023,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
Reply
0 Kudos