VMware Cloud Community
chandlm
Expert
Expert
Jump to solution

Memory Limits

I've been reading up on some details of memory reservations, limits, amount granted, etc. in the documentation and on the forums. I'm still a bit puzzled by something that I've found quite a few posts discussing.

If the a guest will never use more than what is granted, why am I able to set a limit higher than that number? Not a huge deal, just something that is bothering me because I don't understand.

Mark

0 Kudos
1 Solution

Accepted Solutions
esiebert7625
Immortal
Immortal
Jump to solution

It basically defaults to the maximum amount that is in the host (in other words unlimited). Just a quirk of the settings in VC.

An ESX Server host never allocates more memory to a virtual machine than its specified physical memory size. For example, a 1GB virtual machine might have the default limit (unlimited) or a user-specified limit (for example 2GB). In both cases, the ESX Server host never allocates more than 1GB, the physical memory size that was specified for it.

Check out this thread...

http://www.vmware.com/community/thread.jspa?messageID=627962

Also if you want to know more about memory...

Host Mem and Guest Mem % - http://www.vmware.com/community/thread.jspa?messageID=620581&#620581

http://www.stanford.edu/~stinson/cs240/cs240_1/revs/esx.txt

The Role of Memory in ESX Server 3 - http://www.vmware.com/pdf/esx3_memory.pdf

Memory resource management in Vmware ESX Server - http://www.vmware.com/pdf/usenix_resource_mgmt.pdf

Understanding memory usage in Windows 2000 - http://www.microsoft.com/technet/prodtechnol/windows2000serv/tips/w2mem.mspx

Operating System Extensions to Support Host Based Virtual Machines - http://www.eecs.umich.edu/techreports/cse/2002/CSE-TR-465-02.pdf

Memory performance - http://www.vmware.com/community/thread.jspa?threadID=65293

Memory Shares/Limits/Reservations - http://www.vmware.com/community/thread.jspa?threadID=73411

RAM, Virtual Memory, Pagefile and all that stuff - http://support.microsoft.com/?kbid=555223

Understanding the Impact of RAM on Overall System Performance - http://www.windowsnetworking.com/articles_tutorials/Understanding-Impact-RAM-Overall-System-Performa...

RAM, Virtual Memory, Pagefile and all that stuff - http://members.shaw.ca/bsanders/WindowsGeneralWeb/RAMVirtualMemoryPageFileEtc.htm

Pagefile thread - http://www.vmware.com/community/thread.jspa?threadID=79080&messageID=617876#617876

Resizing Memory With Balloons and Hotplug - https://ols2006.108.redhat.com/reprints/schopp-reprint.pdf

View solution in original post

0 Kudos
17 Replies
esiebert7625
Immortal
Immortal
Jump to solution

It basically defaults to the maximum amount that is in the host (in other words unlimited). Just a quirk of the settings in VC.

An ESX Server host never allocates more memory to a virtual machine than its specified physical memory size. For example, a 1GB virtual machine might have the default limit (unlimited) or a user-specified limit (for example 2GB). In both cases, the ESX Server host never allocates more than 1GB, the physical memory size that was specified for it.

Check out this thread...

http://www.vmware.com/community/thread.jspa?messageID=627962

Also if you want to know more about memory...

Host Mem and Guest Mem % - http://www.vmware.com/community/thread.jspa?messageID=620581&#620581

http://www.stanford.edu/~stinson/cs240/cs240_1/revs/esx.txt

The Role of Memory in ESX Server 3 - http://www.vmware.com/pdf/esx3_memory.pdf

Memory resource management in Vmware ESX Server - http://www.vmware.com/pdf/usenix_resource_mgmt.pdf

Understanding memory usage in Windows 2000 - http://www.microsoft.com/technet/prodtechnol/windows2000serv/tips/w2mem.mspx

Operating System Extensions to Support Host Based Virtual Machines - http://www.eecs.umich.edu/techreports/cse/2002/CSE-TR-465-02.pdf

Memory performance - http://www.vmware.com/community/thread.jspa?threadID=65293

Memory Shares/Limits/Reservations - http://www.vmware.com/community/thread.jspa?threadID=73411

RAM, Virtual Memory, Pagefile and all that stuff - http://support.microsoft.com/?kbid=555223

Understanding the Impact of RAM on Overall System Performance - http://www.windowsnetworking.com/articles_tutorials/Understanding-Impact-RAM-Overall-System-Performa...

RAM, Virtual Memory, Pagefile and all that stuff - http://members.shaw.ca/bsanders/WindowsGeneralWeb/RAMVirtualMemoryPageFileEtc.htm

Pagefile thread - http://www.vmware.com/community/thread.jspa?threadID=79080&messageID=617876#617876

Resizing Memory With Balloons and Hotplug - https://ols2006.108.redhat.com/reprints/schopp-reprint.pdf

0 Kudos
bertdb
Virtuoso
Virtuoso
Jump to solution

a host will use a certain amount of "machine memory" to store the guests memory. That amount will never be more than the amount of guest memory plus the overhead (and the overhead depends on the type of VM, the amount of memory, 64 bit or not)...

0 Kudos
msgertz
Enthusiast
Enthusiast
Jump to solution

A virutal machine has three settings that affect is memory resource allocation: memory limit (measured in MB), memory reservation (measured in MB) and memory shares. Reservation is the amount of memory guaranteed (reserved for this vm) and limit is not guaranteed ( a cap on the consumption of memory). Shares control how often the virtual machine wins competition for memory when memory is scarce.

0 Kudos
chandlm
Expert
Expert
Jump to solution

So maybe I did understand but didn't know it Smiley Happy

There is basically no effect to the guest by changing that limit anywhere above the assigned memory, correct? I guess if that is the case it was just confusion as to why it would even be an option.

Mark

0 Kudos
esiebert7625
Immortal
Immortal
Jump to solution

And the overhead is not accessible or seen by the VM in any way though. You can view the overhead on the performance tab by selecting the memory counter Memory Overhead

0 Kudos
esiebert7625
Immortal
Immortal
Jump to solution

That is correct, it's a confusing setting as you'll see in the previous post I referenced....

0 Kudos
chandlm
Expert
Expert
Jump to solution

Right but as posted above (and from one of the docs I read):

An ESX Server host never allocates more memory to a virtual machine than its specified physical memory size. For example, a 1GB virtual machine might have the default limit (unlimited) or a user-specified limit (for example 2GB). In both cases, the ESX Server host never allocates more than 1GB, the physical memory size that was specified for it.[/i]

Then it isn't really a limiting factor for the memory if it is set above the granted memory and is sort of a quirk as esiebert memtioned above. Basically for the purpose of a Guest assigned 1 GB of memory, the unlimited checkbox may as well just say '1 GB' correct?

0 Kudos
esiebert7625
Immortal
Immortal
Jump to solution

Exactly, confusing I know. Maybe it was harder to code into the VI client that way...

0 Kudos
muratgunes81
Contributor
Contributor
Jump to solution

Hi everbody ı have probleblem about RAM tuning in ESX 3.0.

I created 3 virtual machine. And ı want that, All virtual machines uses RAM dinamically in a RAM pool. For example I have a server with 4GB RAM. with ESX 3.0 and 4 virtual machine (1GB RAM per vm).

So when a virtual machine needs extra RAM , then it will take RAM of other vms ,

ex:

VM1 =1GB RAM -


needs 1,5GB ??

VM2 =1GB RAM (in use 800MB)

VM3 =1GB RAM (in use 900MB)

VM4 =1GB RAM (in use 500MB)

I want that vm1 goto other vms and uses their ram (temproray ) then end of need give it again them.

How do I do this configuration in ESX 3.0

Can you help me please ?

muratgunes81@gmail.com

0 Kudos
TomHowarth
Leadership
Leadership
Jump to solution

you can't, you can oversubscribe memory, ie 4Gb physical, each VM has 1.5 GB Memory therefore 6GB but you cannot dynamicaly assign memory to a VM, just like you cannot dynamically assign memory to a physical machine

Tom Howarth VCP / VCAP / vExpert
VMware Communities User Moderator
Blog: http://www.planetvm.net
Contributing author on VMware vSphere and Virtual Infrastructure Security: Securing ESX and the Virtual Environment
Contributing author on VCP VMware Certified Professional on VSphere 4 Study Guide: Exam VCP-410
0 Kudos
muratgunes81
Contributor
Contributor
Jump to solution

if yes about Dynamicaly assign memory, it will be very good.

0 Kudos
RParker
Immortal
Immortal
Jump to solution

Ever tried resource pools? This would solve your problem.

You can create a pool, say 2G.

Put all 4 VM's in it. What's the likelihood that ALL of your VM's NEED 1G all at once? Probably very low..

In a larger server farm, more VM's = lower likelihood that ALL of them are going to be using their full compliment of RAM allocation, pools help fix this problem, and a pool is dynamic.

The memory size that you set is a limit. A VM cannot go over that limit. Since you can't dynamically allocate memory while the VM is running, you can simply allocate as you did enough RAM and then use pools / user limits to keep the memory at CURRENT usage, not future usage.

You have to manage your ESX server, look at the performance chart, and keep an eye on what EACH VM is doing. ESX requires some management, you can't expect to just set it and walk away. Any good system requires management.

You can use tools like ESX charter to plot your data and export it to a PDF, since ALL VM's have different requirements, its safe to assume their memory limitations will be different as well.

IF VM1 has 1G physical allocation, yet the avg memory never goes above 512, you are wasting resources. You should be able to determine how much a given VM is USING and watch the peaks to ensure adequate memory, not fluff.

This why I ALWAYS start low on my memory, and wait for people to complain it's not enough. In 6 months, I have only had to increase memory from 512 to 768 on just 2 VM. You may even find less is more, try setting your physical values a little lower.

4 VM's @ 1G each is your host limit. 4 VM's @ 768 is 3072 leaving approx 1G for 1 more VM . . . . . . Managment is the key.

0 Kudos
RParker
Immortal
Immortal
Jump to solution

It's not confusing you are reading too much into it.

It's a setting. Change your resolution of your desktop to extend to multiple monitors, say 1920 x 1440. Your PHYSICAL monitor can only be 1280 x 768. You save those settings. You buy a new video card, that can now do 1440 x 1280. How is the software going to know that you new monitor (specified in the settings file) will or will not support this new resolution unless you tell it?

Same with ESX. It's a setting in a file. Reverse your situation. Maybe initially you set your memory to only 768, with a user upper limit of the same. Now you later change the physical RAM to 1G. What's the memory of that VM? 768 (as specified by the user upper limit).

That's what that little article is explaining. Maybe you have a template where ALL VM's start from, and you pre-define your upper limits.. ah ha! didn't think about that did you?

You don't want someone giving 2G to a VM and all of a sudden you have a resource problem, that's all these tools are designed to do, MANAGE your VM's. It's not brain surgery. Upper limits are different than physical parameters (those are just starting points anyway) because you have to give the VM something.

A VM is an OS, the OS does not know it's a "virtual" machine, and you wouldn't swap your RAM in your computer at home and all of a sudden take 1g out and replace it with 2g -not while it's running-.. Windows, Linux, and all those other OS models weren't designed with "dynamic" RAM in mind.... VM Ware has found ways around these limitations, but you MUST give the OS something to use at the start, that's why you set a physical number for RAM.

0 Kudos
smccreadie
Enthusiast
Enthusiast
Jump to solution

I'm not sure I understand (still)...

Why have two methods of effecting the upper limit of memory (vmx config and "Resource Limits".

Why not just change the VM config to the correct amount of memory that the machine needs and leave the resource limit to "unlimited"? What is the fundamental difference?

0 Kudos
smccreadie
Enthusiast
Enthusiast
Jump to solution

I just looked at the resource management guide - it says you can use it to "manage user expectations" by chaning it to a lower value to see how things run...

Sounds like a cheesy reason. There has to be a better reason.

0 Kudos
tlyczko
Enthusiast
Enthusiast
Jump to solution

Question about resource pools here.

I'm still learning here, I will have 3-4 Windows Server 2003 VMs in an 8GB ESX Starter box and start with say 768 per VM, then watch the utilization.

Can I later switch to using resource pools??

Would it be sensible to set up a 7GB resource pool and let the VMs take what they need?? Or would this cause too much resource contention among the busiest two servers and shortchange the two less busy servers??

I don't intend to go over 4 VMs in this box due to having to use local storage and the 8GB limit on ESX Starter.

VMware PLEASE remove the ESX Starter memory limitation!!!

Thanks, Tom

0 Kudos
vmkrtg
Contributor
Contributor
Jump to solution

I think you can use shares (high, normal, low) to distribute your memory. So if you give more shares to you busy vm, and less to the other ones; the busy one will take unused memory from those less busy one and return it when done.

0 Kudos