Resource settings (upgrade from 3.0.2 to 3.5u1)

[cxo]

An observation I had when deploying a Windows template...

Deployed an already existing (and heavily used) Windows 2003 template. After deploying, the vCPU was changed from one to two and memory was increased from 384 MB to 2 GB. Powering on created a very slow boot (read 1 to 6 minutes). A reboot would take from two to twenty (or more) minutes. Did plenty of searching both here and on the net and everything pointed to a Windows issue when going from 1 CPU to 2 (virtual or physical). Plenty of suggestions were made including needing to create a new template based on a multi-processor base.

Did some more digging as we have deployed other 2 vCPU systems from this initial template before without incident. Other test VMs were upped to 2CPUs and rebooted in 15 seconds. I then did some more mining and comparing with my test VM and newly deployed VMs and ultimately found the correlation in the .vmx file. It was shm.mem.max. New deployments were setting this to the original 384MB. Older VM didn't even have this configuration present. Did some more digging on shm.mem.max and found this should be set more appropriately (or "unlimited"). Did this but on reboot it always got reset to 384MB and performance on these beefier VMs were still sluggish.

Did some more checking and noticed that under VM resources (Edit Settings --> Resources tab) the CPU and Memory Resource Allocation Shares were changed from "Normal" to "Custom". In addition the Memory "Unlimited" check box was now unchecked (and 384MB was in the Limit box). Fixing these values solved my problem. Now, going back to all my other existing VMs I see the same issue (fortunately they are all running with this episode not exposed). I am manually selecting the correct values (stupid grunt work, but not too burdensome).

It appears this occurred when we upgraded from ESX 3.0.2 to ESX 3.5u1 and/or VirtualCenter 2.0 to 2.5.

Question: Is this a known issue during such and upgrade that I simply missed or is it completely unrelated and expected behavior or a completely other issue. Any one else seen this?

[Texiwill]

Hello,

Moved to the VI: Virtual Machine and Guest OS forum

I had similar issues but I am not sure if it was due to the upgrade or not. Some VMs had odd settings recently and some did not. I now make it a point to check shares after updates, vMotions, etc.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[kratclif]

Hello,

Did you ever find a resolution to your issue? We are running VC2.5u3 with ESX3.5u3 and have a similar problem with a 32-bit Windows template. We can convert the template to a VM, check "unlimited" for CPU and memory resources, convert back to template, and every time we deploy from this template the new VM has a memory limit set at 384MB when the guest has 1024MB. Next step is build a new template I guess...

Thanks for any information!

[cxo]

it appears that sometimes it works, other times it does not. Of course enough time passes between issues and I forget what to check.

[caldwelr]

I'm assuming that the original poster meant to say sched.mem.max rather than shm.mem.max.

Bumping this up because we've just discovered this same issue after migratiing directly from ESX 3.0.2 and VC 2.0.1 to ESX 4.0 and vCenter 4. Many of our VMs now have a memory limit set the same as the allocated memory. Of course this is harmless unless you increase the memory on a VM and fail to notice that there's a limit on the resource. But it's a trap that's easy to fall into. If you never set memory limits you tend not to check them when increasing memory allocation.

Unfortunately I don't have any 3.0.2 systems left to test this and determin when the limit was set. Based on when what I've been able to determine it appears to have happened either when we moved our 3.02. hosts under vCenter 4 or when we vMotioned our VMs to new ESX 4 hosts. Based on our timline it appears to have occured before we upgraded vmware-tools and the VM hardware version.

If you migrated through a similar process be sure to check for memory resource limits.