ESXi 4.0 really slows down with only 4 minimal virtual machines running

[Steve_Hengen]

My ESXi Server is a Quad Core single processor (2.4GHz) Server with 8GB or RAM

It runs great with any three Virtual machines it seems, but when add the fourth they all slow to a crawl.

There are many tabs that don't make sense for example:

I look at the Resource Allocation tab it shows CPU Capacity 8149 MHz - So I assume it's using 1.5GHz which is a lot, but I can live with that, but the Memory says the total capacity of the RAM is 1572 MB - So the system is using over 6GB of RAM? What's the deal?

I look at the Configuration Tab - Hardware Memory and it says Memory: Physical Total 8114 MB, System 5288 MB, Virtual Machines 2826 MB

Currently there are three machine running - one I have limited to 512MB (Currently using 52MB and the Host is using 127MB), another is limited to 1.5GB but it is using only 600MB (Host is using 1014 MB) and the has no limit, but has only 1GB allocated (It is currently using 80MB and the host is using 913MB)

BTW Machines 1 and 2 above are limited to 500MHz as well and Machine 3 is limited to 1200MHz.

So what in the heck is the deal? Where are all of my resources going? My VM's aren't using hardly anything. If I try to start a 4th VM that is limited to 512MB and 500MHz,all of the machines become unusable. Network connections start dropping off, RDP can't connect, the console window is only good for trying to shut it down. As soon as any of them is turned off, it runs fine again.

I have trying tons of settings, from setting everything to unlimited and forcing them all to unreasonably low speeds and memory and nothing works.

Where do I start to figure this out? I've been dealing with this for over 6 months and am very frustrated.

[Rumple]

First thing I would do (since you don't need to do this on your environment) is get rid of any resource pools, memory share changes or any CPU limits you placed on the VM's and make sure all your VM's are single CPU

This will help us track down what is going on by being a standard general configuration. Then you can with limits etc

with a single quad core CPU with no limits you will easily run 12 VM's or more for testing...assuming your Disk is on a RAId controller with a Battery Backup (BBWC) and Write back enabled...else your disk performance will probably cause you a load of problems...

[J1mbo]

Agree with the above. TBH I'd be tempted to do a 'clean' ESXi install on it and get the RAM issue sorted before adding VMs back on. I seem to recall this came up before, and a BIOS update resolved the RAM issue.

Presently most likely there is heavy vSwapping due to lack of available ram, which affects performance horribly.

Please award points to any useful answer.

[bulletprooffool]

I am not sure why you have set any limits on these VMs?

You have 4 VMs on a Quad core processor - so in theory, each VM has its own vCPU (each core becomes a vCPU to the VMs)

Removing limits in your instance will notreduce the performance of any VM . . no matter how hard you run the other VMs.

In addition, you should remove any memory limits and simply only allocate the amount of memory each VM needs. If your VM is using more memory than you have set as a limit .. it will be writing to physical disk . . that it thinks is RAM (hence a huge performance hit)

[Steve_Hengen]

This is a very basic install, the only reason I tried limiting the CPU and memory was to try and get better results - but all of my attempts have failed.

I am going to move the VMs to another server, reinstall ESXi and try again. Unfortunately my testing with the VM's isn't complete, so I can't just blow them out.

But to stop this from happening again - Why would the Resource Allocation Tab show Total capacity 1575MB, but the Configuration tab show 8114MB?

I will report after I reinstall and retest.

[Steve_Hengen]

So, I reinstalled ESXi 4 but a clean install still shows only 1.6 GB or RAM available of the 8GB installed. It says the system is still using over 5GB.

Why?

I look at the Configuration Tab - Hardware Memory and it says Memory: Physical Total 8114 MB, System 5288 MB, Virtual Machines 2826 MB but there are no virtual machines on it yet.

[Steve_Hengen]

I did a clean install of ESXi 4, but the same results.

I look at the Configuration Tab - Hardware Memory and it says Memory: Physical Total 8114 MB, System 5288 MB, Virtual Machines 2826 MB, but there are no machines on it yet, and it says there is only 1.6 GB of RAM available

[Rumple · Accepted solution]

You may want to put the latest bios on that system as well as check to ensure bios defaults are set (with exception of vt stuff

[Steve_Hengen]

Actually I had to move to an older BIOS! The latest BIOS for my board seems to have been the problem with the RAM