ESX 3i - Why is all my Ram Reserved?

[TheSmokey]

I have setup ESX 3i on a spare machine I have here (AMD 3000+ (2ghz), 1GB Ram). I had setup two VMs, one Linux VM for an ubuntu with 512MB allociated to it, and an empty VM with 128mb of ram allocated (but neither had Ram 'reserved').

After doing a partial setup of the Linux vm I found out that when attempting to start the second VM it would error out saying that I don't have enough ram avaiable to run it (sorry, I forget the actual message).

Investigating the issue I found that half my CPU is reserved and I have only 103MB of ram reservable. After finding nothing that would explain why this is happening I reinstalled with the latest updated released and still these reservations remain. Can anybody explain why 900mb of my ram and half my CPU is reserved like this? Is this normal for ESX and I just need more RAM? Even still why can't I run two VMs I thought you could oversubscribe RAM usage?

Here are two screenshots after I had reinstalled. First showing that memory and cpu is being reserved on a clean install, and second showing my ram and cpu being detected. Thank for any help

http://intothecloud.net/i/vmware1.jpg

http://intothecloud.net/i/vmware2.jpg

[Dave_Mishchenko]

Welcome to the VMware Community forums. There's an issue with the overhead that ESXi takes to manage the VMs. What you can do to free up some more memory is to go to Configuration \ System Resource Allocation. Then select advanced, find the VIM object and set it's reservation to 0. This should bring you up to around 500 MB on the resource allocation tab.

I've spoken with a person in support and he's looking at the issue. ESX / ESXi have to allocate memory to manage your virtual machines and while ESX would take about 200 MB, ESXi seems to take about 700 MB.

[wizdude]

I would highly recommend to upgrade the memory. I've got 2GB in my box and it behaves a lot better.

for some reason the overhead in ESXi is greater than ESX (and potentially this might be a bug). Upgrading the RAM is an easy solution, especially given the fact that RAM is cheap.

[leofonic]

Welcome to the VMware Community forums. There's an issue with the overhead that ESXi takes to manage the VMs. What you can do to free up some more memory is to go to Configuration \ System Resource Allocation. Then select advanced, find the VIM object and set it's reservation to 0. This should bring you up to around 500 MB on the resource allocation tab.

I've spoken with a person in support and he's looking at the issue. ESX / ESXi have to allocate memory to manage your virtual machines and while ESX would take about 200 MB, ESXi seems to take about 700 MB.

The difference between total memory installed and available memory for resource pools ("Memory Reservation") is even 1,5GB. With 1,5GB physical Memory installed, i'm not able to start a single virtual machine because there's only 60MB available for virtual machines (as shown in "Memory Reservation"). The trick with editing reservation for the VIM object allows me to start a VM, but i have to do this again after host restart.

[Dave_Mishchenko]

What sort of reservation was VIM asking for by default. On all my systems it has been 512, but some users have post it as asking for 1024 MB.

[leofonic]

I have two systems i can look at now, both have reservation and limit set to 1024MB on the VIM object by default.

System one has 1,5GB memory installed and 60MB left in "Memory Reservation".

System two has 2GB installed and 500MB left in "Memory Reservation".

[wizdude]

i would be interested to know if this still exists in post-aug12-update2 (build 110180). i've just added up the memory taken up by my VM's and it all checks out fine.

after reboot, memory usage shows as 358MB. I boot up a windows 2003 r2 VM which reports host usage as 288MB and the memory usage reports as 646MB.

bring up a linux VM which reports host usage as 129MB and the memory usage reports as 802MB.

i have applied no workarounds to my ESXi host at all.

i have just fired up 8 VM's on this box now (half windows, half linux) - memory utilisation is 1.46GB.

[wizdude]

i would also be very interested to see your esxtop memory results.

enable ssh on the console, connect in remotely and run "esxtop" then press "m" for memory

post up the screenshot/results.

[leofonic]

My ESXi says it's build 110271 (all patches applied via update manager).

The numbers you mention really do match (at least roughly):

"Summary"-tab of the host shows something like 350MB, plus the memory taken by virtual machines.

"Memory"-item in "Configuration"-tab of the host shows total Memory and about 250MB less for Virtual Machines.

So far everything is fine, BUT:

"Memory Reservation" in "Resource Allocation"-tab of the host shows roughly physical memory minus 1,5GB (or something less than 100MB on machines with less than 1,5GB physical RAM). Unfortunately this is the only memory usable by VMs, as far as i can see. If they need more memory ESXi starts to swap. Additionally, a VM needs about 90MB Memory Overhead from reserved memory. This explains why you can't start 2 VMs with 1GB RAM, Memory Overhead would be 180MB, which is more than shown in "Unreserved Memory" (about 90MB with 1GB physical RAM).

I know this can be solved by adding memory, and a production machine will more likely have 8GB than 1GB RAM, but this doesn't seem the way it should be anyhow.

I just read your last post, i'll try esxtop as soon as i have the time.

[wizdude]

i've just connected to my ESX 3.5 box at the office ... it's almost the identical hardware to my ESXi 3.5 box at home ...

both boxes have 2GB of memory ... ESX 3.5 shows a memory reservation of 1456MB. ESXi 3.5 shows a memory reservation of 1041MB.

this means ESX is 400MB better off ... enough to launch another VM (or perhaps two) in 1GB of memory.

i agree this is a problem, and I hope we hear about this as a bug report/fix. as soon as you go to add more memory, however, it becomes less of an issue.

just pretend the ESXi box is running windows! you can't run Windows 2003, Virtual Server and Three VM's in 1GB of memory either :smileylaugh:

[leofonic]

Just tried esxtop, the free memory shown is the same as in "Summary"-tab (1677MB free on a host with 2GB RAM and no machines running).

I did some more testing and have to correct myself about the memory usable by VMs. I got the impression VMs could only use "Reserved Memory" because:

a) I started a 512MB VM on a host with 1GB and the host started swapping.

b) You can't reserve more memory than the value in "Reserved Memory" (of the host) for VMs and resource pools.

But right now i started a machine that is assigned (and really uses) 1GB on a host with 2GB RAM (500MB show up in "Reserved Memory"), and no swapping occured. So the "Reserved Memory" is appearantly only needed for Memory Overhead, which limits the number of VMs you can start. So it seems on a host with 2GB or more, everything is fine as it should be, but on a host with less than 2GB, "Reserved Memory" is too small to provide the Memory Overhead needed to start multiple VMs.

[wizdude]

i think my next ESXi box will have 4GB of memory in it ... it's getting so cheap these days

a comparison between ESX and ESXi shows that vim has both a 1050MHz CPU and 512MB MEMORY reservation, whereas ESX has no such reservation.

this is obviously a design difference (and i'm guessing its related to the service console), since the processes handled by vim change between ESX and ESXi.

[wizdude]

it's also possible to edit the startup memory reservation for vim.

ssh into the ESXi host and edit /etc/vmware/esx.conf

/resourceGroups/root/child(0000)/mem/minimum = "512"

(please note real line has square brackets but i cannot use them here)

i'm not sure how this original amount is calculated since hosts some have 512 and others have 1024.

i'd probably recommend this to be set to 256 and not zero

again, more memory is a good guideline here as well.

[leofonic]

You're too fast for me, i'm always 1 post behind :D.

This explains the difference in Reserved Memory between your machine and mine, my box has a reservation of 1024MB on the vim object. I'll try if it's possible to set it to 512MB or less permanently.Otherwise i'll go out and get some more RAM. Maybe they should just raise the specs to 2GB minimum RAM needed.

Now i'm 2 posts behind. Thanks for pointing out how to change the reservation for VIM, i'll try this.