ESXi 5.5 cluster - seeing high amounts of swapping on random set of VM's

[JCMorrissey]

Hi all,

Got a 26 node cluster (which is running a mixture of 5.5 builds which does need to be addressed) and see certain number of vm's swapping out at time (under resource utilization tab it can be seen).

In terms of memory over-commitment there is not a lot of it - on average each host has 256GB of RAM and maybe 265-270gb of RAM assigned to VM;s (was supposed to be 1:1 ratio but some additional vm's got added). Thing here is the active guest memory usage never spikes too high (maybe 10gb after the reboots have settled down) so looking at roughly 20-40% actual utilization. Just wondering if folks had any pointers to assist here.

tx

[Nick_Andreev]

"Active" memory is what the VMkernel believes is currently being actively used by the VM. You should be looking at the "Consumed" memory. Plus bear in mind that it doesn't include VM memory overhead. Depending on which level you are looking at, it may also not include the ESXi host memory overhead.

Could you post the amount of "Consumed" memory from the "Host" level?

[pkolom]

Few hints I can think of:

- Are VMtools installed on VM's? Swapping is last resort, should be balooning and compressing first...

- How many VMs/host you have? It may be memory overhead if you have lots of VM's CPUs/VM or lots of memory in VMs..

- any reservations on VMs/resource pools? If yes are they needed?

[JCMorrissey]

Enclosed above the memory consumed.

Yes good point there on the memory overhead - we do run a standard of 8 VMs (32gb RAM) on a 256bg RAM box so yes there will be some overspill for sure.

Other piece was reservations - for some reason some had them in and been removed.

Last piece is the VMware tools - so yep got a mixed line of builds (hypervisor team hopefully getting them in alignment next wk) but getting the vm's up-to-date will be a little tricker as using citrix pvs imaging software and need to reverse engineer the images to get them up-to-date but will be on the list.

[pkolom]

Screenshot shows balooning rate  = 0 that means that something is wrong with baloon driver in VMs. This driver is vmtools component, so correcting this is your first step. ESXi 5.5 allows updating tools without VM reboot, so you could update tools with update manager and not need to reverse engineer your images.

[Nick_Andreev]

Also, as you can see in the "Consumed" stats, your VMs consume the amount of memory very close to your physical host configuration, which is 256GB. Add some VM memory overhead and you hit the limit.