ESX4 + Nehalem Host + vMMU = Broken TPS !

[mcwill]

Since upgrading our 2 host lab environment from 3.5 to 4.0 we are seeing poor Transparent Page Sharing performance on our new Nehalem based HP ML350 G6 host.

Host A : ML350 G6 - 1 x Intel E5504, 18GB RAM

Host B : Whitebox - 2 x Intel 5130, 8GB RAM

Under ESX 3.5 TPS worked correctly on both hosts, but on ESX 4.0 only the older Intel 5130 based host appears to be able to scavenge inactive memory from the VMs.

To test this out I created a new VM from an existing Win2k3 system disk. (Just to ensure it wasn't an old option in the .vmx file that was causing the issue.) The VM was configured as hardware type 7 and was installed with the latest tools from the 4.0 release.

During the test the VM was idle and reporting only 156MB of the 768MB as in use. The VM was vmotioned between the two hosts and as can be seen from the attached performance graph there is a very big difference in active memory usage.

I've also come across an article by Duncan Epping at yellow-bricks.com that may point the cause as being vMMU...

MMU article

If vMMU is turned off in the VM settings and the VM restarted then TPS operates as expected on both hosts. (See second image)

So if it comes down to chosing between the two, would you choose TPU over vMMU or vice versa?

[Ajay_Nabh]

Hi Guys

I am going thru the same issue as you guys. however as it been mentioned in Blog "If memory contention develops, the VMKernel will automatically switch to small pages and implement TPS in an effort to free up memory"is working nicely and kicked in after few days of me setting up cluster. this may give some people consolation, so i thought

[AndyR8939]

If I enable the advanced setting, do I then need to vmotion all my vms from the host so the host is empty before vmotioning them back, or can I just vmotion the vm out and in again then just repeat for each VM?

Just thinking best way to do it because if I have to vmotion every VM from the affected host I'm going to need to shut some down I think.

Thanks.

[Avial]

I got this error last year whan i upgrade from ESX 3.5 to 4.0 setting the AllocGuestLargePage to 0 did the trick.

Now i'm upgrading to ESXi 4.1 U1 i'm getting high memory allocation with no guest memory usage.
This time nothing seems to work...
Setting le LargePage to 0, vmotion in and out. still getting the same trouble

[AartK]

I am on ESXi 4.1 and when I vMotion my VM's from DL380G5 to G7 the memory consumption goes up and my private guest memory is 100%.

When I vMotion back to the G5 the TPS kicks in again. yes vMMU will leverage the Large Pages and so TPS will not "function" until there is memory contention on the host.

What about this issue on vSphere 5? Can anyone test this?