Hello,
we are running into severe performance problems on our primary ESXi
production server (Dell PowerEdge 1900 ESXi 3.5.0 110180).
Once our Linux VMs have run for a day on the ESXi server, their
performance die. Large RAM allocations are performed at a rate of 500
KB/s and disk I/O done with 'dd' sometimes slow down to 2.2 MB/s even
when the disk is otherwise idle. Our VMs are mostly idle ('top' reports
>95% idle).
We have made a test program that allocates and initialize a large amount
of RAM. The initial run of the program allocates 256 MB in about 500
seconds. The subsequent run can do the same in 2 seconds. The ESXi
performance monitor shows little disk access (< 1MB/s) during those
tests, so swapping is not the issue. Rebooting the VMs fix the problem
for a while, but the performance degradation comes back after a day.
We have enough RAM on the ESXi host for all our VMs. Write performance
with 'dd' while logged with SSH on the ESXi host is over 50MB/s. The
average CPU usage on the ESXi host is 11%. Our VMs do not exibit the
problem when they are running on a ESX server or another ESXi server.
Our VMs are not running in snapshot mode.
Does anyone have a clue what is going on?
Thank you