Greetings to all.
I'm currently running ESXi 3.5u3 on several boxes and I've noticed some strange behavior with the memory overhead relating to a few Linux based VM's when VMI Paravirtualization is enabled.
These VM's are vastly the same and are all based off a single core image (so the underlying OS is identical).
They are running Gentoo Linux (x86), with a 2.6.x kernel, VMI support enabled in the kernel config and under the Virtual Machine configuration (Paravirtualization). The guest OS is set to "Other Linux (32-bit)". Open-vm-tools has been installed in the VM's, ESXi reports the VMware Tools as OK, along with the hostname and guest OS IP's. Guest shutdown/restart works just fine.
Each VM is running with one vCPU and 128MB of RAM.
After 24 hours of operation (and these VM's idle 99% of the time), the Memory Overhead for each VM was anywhere from 650-800MB (!!!) and slowly expanding with no signs of stopping. On a VM with 128MB of RAM, this is ridiculous. "Guest Memory Usage" is around ~10-37MB per VM, and "Host Memory Usage" is anywhere from 256MB to 560MB.
If I reboot these VM's, all the values reset to what I'd consider normal. Memory Overhead drops to around 75MB (but starts to increase), Guest & Host Memory Usage are both sub-100MB. This begins to increase slowly- by about 0.20MB every few minutes, and doesn't stop.
Disabling VMI Paravirtualization seems to have fixed the problem (I think? I just rebooted the VM's with VMI disabled).
Is this a known issue or is there some sort of memory leak associated with VMI paravirt?
I've noticied exactly the same thing, which I find very, very strange. In Xen, it works the opposite way - HVM domUs incurr a small overhead (512MB HVM domU = ~520MB host usage) whereas PV domUs are always exactly what you assign to the domain. I've not only seen it in ESXi, but all the full ESX. Very, very strange...I'm not sure why - I'm not sure there will be an answer in this forum for why it does that, I think it must have to do with how they implement paravirtualization in an environment that is designed for full virtualization.