I am experiencing very slow Disk I/O performance within my CentOS 5.8 64 Bit guests after I load up my applications.
The server load on the guest remains at around 0.50, and I'm getting the following Disk I/O:
# dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 286.346 s, 3.7 MB/s
However, if I create a new VM Guest with the same OS in the vSphere client, and run this DD test on a NEW guest, it gets the following (very good) results, but after I load up my applications (even though they use very little cpu and iowait, less than a 0.50 server load) it becomes very slow.
# dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 4.51633 seconds, 238 MB/s
What could be the problem? I am using a 4x drive setup in hardware RAID10. I'm using LSI 9260 with BBU.
Additionally I should note that my usage according to the vSphere client on the guest that is experiencing slow Disk I/O is using just 700 MHz of CPU, so its not like my application is io or CPU intensive. I'm using an Intel E3-1240 CPU.
I would appreciate any help as this is very frustrating.
