The thing to remember about how ESXi goes about running VMs is that it dispatches vcores (which basically each represent a physical core on a physical CPU), using a time slicing algorithm, not an interrupt driven algorithm as is typically used to dispatch processes in a regular operating system. What this means in practice is that if you assign say 2 vcores to a guest, then 2 vcores must be available before that guest can run at all, and also once it starts running (i.e. getting its time slice) those two vcores (and hence the two physical cores) are not available to any other VMs until that guest has finished it's time slice, even if the guest is mostly idle during it's time slice. Therefore you have to be careful not to assign a significant percentage of the total vcores available on your system to a specific guest. If you do, you will start to notice choppy performance, even though it doesn't look like the CPU is overcommitted.
As an example, before I understood how this really worked I thought that ESXi worked just like an interrupt driven operating system and that all I had to do is assign the right priority to the workload, and then I could assign as many vcores as I thought a particular VM might ever need. In my case, on a 4 physical core system I assigned 4 vcores to an SBS 2008 VM, and a single vcore to a couple of other WinXP VMs. I noticed that the SBS 2008 VM would always run very choppy when I logged on to it from the RDC console. The reason why was because before the SBS 2008 VM was allowed to run at all, all of the vcores had to be available, which of course meant no other guest could be running, and ESXi itself had to get out of the way too so that the guest could have all the CPU resources in the computer.
After going back and forth with some folks in this forum I finally understood that ESXi was unable to use vcores that were assigned to a running VM, that just happened not to be using them all at the current time, and assign them to another VM at the same time, letting them share the vcore via an interrupt driven strategy. When I cut the number of vcores to two for the SBS 2008 VM, things ran much better, even though it took longer for it to start up and shut down (the only time it really could make good use of all four vcores was during startup, shutdown, and when patches were being applied). After a little more playing around I found that the best compromise was to reserve one vcore for the SBS 2008 VM exclusively (by using the Scheduling affinity to prevent the other WinXP VMs from ever using one of the four vcores) and let it compete for one of the other three vcores with the WinXP VMs using the time slicing parameters. As the SBS 2008 VM was the most important workload on the machine, it made sense to reserve one fourth of the CPU resources in the machine for its exclusive use.
Later on, I transferred the SBS 2008 VM, and the WinXP VMs, along with adding a few more Win 7 VMs, to a new Dell 2970 server which had two physical CPUs, each with 6 cores, which resulted in 12 vcores being available. I was then able to reset the SBS 2008 VM back to using four vcores instead of two, because on a twelve vcore system that only represented one third of the available vcores. I was also able to set the WinXP and Win 7 VMs to two cores instead of one, as that only represented one sixth of the vcores available. I also used the same trick I did on the previous box and reserved two of the twelve vcores for the exclusive use of the SBS 2008 VM so that it only had to compete with the desktop Windows VMs for two of the four vcores it needed to run.
This also highlights the fact that it's better for ESXi, if you are going to run several VMs on it at once, to have a larger number of slower vcores, than a fewer number of faster ones, even if the total computing power of both configurations is theoratically the same (unless you are running VMs that do something like scientific calculations in a single threaded manner) because of how the time slicing works.
mudtoe
