I'm trying to find the best way to assign resources to a couple of VMs under ESXi 5.1 and want to better understand how an "idle" processor is treated.
I read ( http://www.vmware.com/files/pdf/techpaper/VMW_vSphere41_cpu_schedule_ESX.pdf ) that the hypervisor is able to schedule a VM even if not all vCPUs can be assigned to pCPUs at the same time. Basicly it just needs to keep accumulated skew between vCPUs within reasonable boundaries.
I wonder how a vCPU running a thread that waits in a select() on a UDP socket file descriptor is handled. We receive data with ~60 MB/s (i.e. ~40k packets per second, or approximately 25us "idle" time in select()).
Must the vCPU be assigned to a pCPU while it waits? Or does the hypervisor somehow skip that "sleep time" (i.e. don't calculate it into the skew)?
My stomach feeling is, that this vCPU should eventually go into "IDLE" state and not accumulate skew even if it isn't on a pCPU while the "ACTIVE" vCPUs are on pCPUs. But that could be completely wrong. It's also the question how long it would take until the vCPU goes into such an idle state.
Why do I care? Our IO threads require very good responsiveness - they receive video frames via UDP from a digitizer and need to request retransmission/next frame very quickly. But they only use about 10% CPU (i.e. ~20us sleep in select, ~2us receiving+processing data). In the same VM we have threads with compute intensive tasks - convert received video, search for artifacts. They are not time or performance critical (best effort without any guarantees is good enough). The compute tasks do however use "100%" CPU for several seconds when they run (i.e. they never wait or sleep).
On a standalone Debian machine I can simply define processor affinity such that the IO threads run exclusively (alone + always) on certain cores. They are never delayed by anything and the compute tasks simply make progress as they can.
If I strictly bind vCPUs to pCPUs in ESXi than the least available pCPU will impose delays to all vCPUs (else they would accumulate skew, some vCPUs run at 100% remember). But I don't want delays in my IO threads. So what can I do?
Thanks for any advice,