Virtual_Engine2
Contributor
Contributor

VMs are very slow to respond after a period of inactivity

Hi Everyone,

I've got a VM that performs IIS/SQL services, my end-user for this server has reported that the box is sluggish and requested more memory. I've looked at the performance stats through VirtualCenter and based on it's weekly average for CPU and Memory I have reserved some CPU and Memroy to what I feel is an appropiate value. In this case the server is allocated 1024MB (although I don't see this as relevant and would have prefered to set the minimum required by the operating system) with a CPU reservation of 200Mhz and a Memory reservation of 384MB.

The end-user still complained although the VirtualCenter graphs showed it was not stressed but had some peeks above the average values I had set. After talking to the developer he gave me a URL which tied into the servers application. He reported that the initial connection to the VM first thing in the morning using this URL took 30-45 seconds but any subsequent connections took less than a second. I validated this was in fact true. I thought I would be able to capture the required CPU and Memory settings after running the link and seeing whatever the VM consumed however the summary stats weren't updating in a smooth fashion on normal running and found the VM could be using more CPU/Memory before I used the link than after.

I was forced into increasing the reservation values to cover some of the peaks and found a what I thought to be a reasonable setting of 300Mhz and 512Mb of RAM. It would now reboot and you could still connect using the link immediately rather than waiting for the CPU and RAM usage to settle. However left overnight or for a short period of it reverts back to a delay of 30-45 second to connect. I'm now unsure of how to proceed, I do have a call logged with Dell but they are busy suggesting things like upgrade to the latest ESX Server release (I'm currently on 3.0.1 with some but not all of the patches).

Another thread with the same title suggested using a vmx configuration change to disable memtrim rate, but I can't explain exactly what that does and would like to know why my sizing methodoly did not work.

Any suggestions?

Steve.

Tags (3)
0 Kudos
13 Replies
RParker
Immortal
Immortal

memory trimming is a technique to return unused virtual machine memory to the host machine for other uses. While trimming usually has little impact on performance and may be needed in low memory situations, the I/O caused by memory trimming can sometimes interfere with disk-oriented workload performance in a guest.

VMware uses a page sharing technique to allow guest memory pages with identical contents to be stored as a single copy-on-write page. Page sharing decreases host memory usage, but consumes system resources, potentially including I/O bandwidth.

You may want to avoid this overhead for guests for which host memory is plentiful and I/O latency is important. To disable page sharing, add the following line to the virtual machine configuration (.vmx) file:

sched.mem.pshare.enable=FALSE option

PerformanceGrou
Contributor
Contributor

The "lunch break" or "weekend" effect you describe does not seem to have anything to do with the fact that you're using a VM. With traditional paging algorithms, pages that are untouched for a period of time become candidates for being sent to disk if the OS needs memory for another application. If there are any processes that run in the background while the user goes away for the weekend, they could be kicking out pages from memory.

The response about trimming does not apply to ESX, only hosted products such as Workstation or Server.

Page-sharing can cause a slight increase in physical CPU usage, but it's effect on guest workloads is workload dependent (you should only notice if the physical cpu is pinned at 100% (see esxtop) or if the app is extremely latency sensitive. I'd recommend leaving it on.

0 Kudos
wila
Immortal
Immortal

Hi,

Well i suppose you have checked this already, but i've seen this happen on Windows VMs where under "control panel" > "Power options" the machine was set to go in standby after 30 minutes. The scheme should be always on, with non of the options go into standby mode ever.

Same topic, different solution:

http://www.vmware.com/community/thread.jspa?messageID=681970#681970

Message was edited by: wila

Added link

| Author of Vimalin. The virtual machine Backup app for VMware Fusion, VMware Workstation and Player |
| More info at vimalin.com | Twitter @wilva
0 Kudos
hanlim
Contributor
Contributor

Hi, I have the exact problem as well.. SQL in vmware. We just move our SQL server from physical to VM last week and have some issues on performance site.

Did anyone have any resolution on this? system standby is set to never since first day..

0 Kudos
Virtual_Engine2
Contributor
Contributor

I've been working on the problem for a while now and haven't quite sorted the problem. It appears that VM's CPU deschedule when idle and it's not uncommon for it to take a little bit for it 'spin' up again. The advice I have so far is set advanced settings to make sure CPU keeps spinning, but I haven't had chance to test them yet.

0 Kudos
jhanekom
Virtuoso
Virtuoso

This doesn't seem like a VMware-specific problem at all to me, unless you're severely memory constrained on the server and ESX has used balooning to reclaim some memory from the VM (which, from what you say, doesn't seem to be the case.)

I would get a DBA involved if possible. Profile the SQL transactions that takes place. You'll likely notice that the transactions executed during "startup" are different than those during "normal" response times.

Many ASP and ASP.NET applications cache content and queries for performance reasons. Your developer should be well aware of this. By default, IIS "unloads" an ASP application after 20 minutes of inactivity. You can adjust this interval in the IIS administrator console by getting the properties of the application pool that runs your ASP app.

Try disabling the "Shutdown worker process after being idle for" option and see if the problem disappears. Or set the shutdown period to something short, like 1 minute, and see if the problem becomes more evident.

If it becomes evident that the problem lies in the worker process being shut down and started up again, get your developer to sort out his app! Maybe he has some buggy SQL query in his global.asa or something.

ne0031
Contributor
Contributor

I have this issue with asp based web servers as well. I just enabled the changes and am awaitign test results.

0 Kudos
Virtual_Engine2
Contributor
Contributor

Wait did you set the spin rate too? Or are you looking at the other suggested changes within the OS?

0 Kudos
daniel_uk
Hot Shot
Hot Shot

I experience this when running SAP in a vm, once the vm has "warmed up" it operates acceptably. The guys above me have all explained why this happens, its just Vmware scheduling its resources that is slowing this.

What other Vm's are running on this box?

0 Kudos
AdamSnow
Enthusiast
Enthusiast

We are seeing this issue too in nearly every VM we have - both converted VMs and physical to virtual. When I come in the office first thing in the morning and try to access a VM, it is very slow and seems to take time to warm up. I see this with remote desktop access, websites on the VM, file shares, etc. It seems as if it goes to sleep or something when it's been inactive for awhile. after it "warms up", performance is good.

We are not overcommitting on memory or cpus. It seems like ti must be some setting somewhere that we need to tweak. Has anyone been able to solve this?

By the way it only seems to afect windows VMs.

0 Kudos
AdamSnow
Enthusiast
Enthusiast

Bueller?

0 Kudos
Thorsten_Schnei
Hot Shot
Hot Shot

Hi all,

the thread has been quiet for a while and I'm wondering if there are any news ? I'm also seeing that behaviour on VMs and would like to know how to avoid or solve it.

Thanks

Thorsten

0 Kudos
str1
Contributor
Contributor

I'm experiencing the same issue with vmware esxi 4 and multiple ubuntu servers. I have made many configuration changes (these mem etc. settings) but no help. This is really urgent issue for me so please if someone has a solution to this it would be great to know what to do!

Problem:

  • Ubuntu web server takes 20-60 secs to respond to web client after a while of inactivity

  • Performance log shows high disk usage peak before system returns to normal state

System has 9650 AMCC controller etc. server hardware, so the problem shouldn't be in hardware side. It would also help to narrow down the search to client software if someone has knowledge about the vmware structure: is there possibility that client for example goes to some kind of sleep mode on the vmware side which would trigger this hard disk usage slowdown? If there is something like that, then is it possible to turn this off completely?

0 Kudos