VMware Cloud Community
devzero
Expert
Expert
Jump to solution

Linux guests stutter under load

please also see http://communities.vmware.com/message/533875

i still have no explanation for why this happens and just don`t want this thread to get forgotten.

ok - it`s an issue with SATA disks, but it`s yet to be explained what`s causing the problem.

i see more and more people using sata disks attached to sas controllers (for cheap test boxes) - so i`m wondering if i`m alone with this....

repost:

if i do "dd if=/dev/zero of=/test.dat bs=1024k" inside a linux VM, my VM begins to stutter very soon, "vmstat 1" doesn`t print a line each second but needs at least 5-10 times longer (time seems "jumpy")

after a while (until disk gets full), dd reports a write rate which is quite nice, but only from the VMs point of view. (time is relative, as Einstein tells).

If i measure the real time for that write of ~5GB and calculate the average, write performance is horrible with this (factor 10 less performant from what i expect or what i have seen with booting from knoppix before installing ESX)

this is on a server with integrated LSILogic SAS-controller and IME (integrated mirroring)

i tried this with my home system and could see a similar effect.

my home system has LSIlogic Megaraid 150-4 controller (SATA) and for my surprise i was able to solve this issue (sluggishness and slow write) completely by changing the controllers caching policy from write-through to write-back. works and performs pretty well here at home. i get around native disk speed from inside a VM!

the problem with the SAS system and IME is:

it seems that i cannot tune caching policy of that controller!

http://www.lsilogic.com/files/docs/techdocs/storage_stand_prod/PCISCSICont/Software/ir_ug.pdf

telling:

"The IM firmware disables disk write caching. This is done to increase data integrity, so that the disk write log stored in NVSRAM is always valid. If disk write caching were enabled (not recommended), the disk write log could be invalid.

while this "stuttering" happens, the VM is nearly unusable and timing is completely crazy....

Can someone tell, why this happens?

Can this be tuned somehow ?

Maybe VMFS3.DoSyncDiskActions in advanced settings?

Reply
0 Kudos
28 Replies
devzero
Expert
Expert
Jump to solution

>I am a foreign national and have recently come into a large sum of money, for your help in this transaction....

thanks, but please - could you first prove that you`re NOT from Nigeria ? :smileygrin:

Reply
0 Kudos
beovax
Enthusiast
Enthusiast
Jump to solution

Did you make this change is ESX or the guest vm's??

Reply
0 Kudos
devzero
Expert
Expert
Jump to solution

that change was done in esx (cos) - esxcfg-module is your friend

Reply
0 Kudos
beovax
Enthusiast
Enthusiast
Jump to solution

Sorry to be pain, what esxcfg command di you run to make the change?? I'm writing out your cheque now :smileygrin:

Reply
0 Kudos
devzero
Expert
Expert
Jump to solution

see my last post Smiley Wink

do you have that issue ?

Reply
0 Kudos
beovax
Enthusiast
Enthusiast
Jump to solution

Sorry, i have probably got myself confused, my linux skills are not too great. I have an LSI 150 sata card using the mesgaraid2 module. I followed the LSI instructions

insmod ./mpbase.ko mpt_can_queue = 128

but I cant find the mpbase.ko. Do I need to run the esxcfg-module with the mpt_can_queue command? I know im being stupid, can I blame it on a long day and it getting late??

Reply
0 Kudos
devzero
Expert
Expert
Jump to solution

you can list with esxcfg-module which modules are user by esx kernel. (don´t have esx here, so i cannot tell you exact syntax)

as you are using different module, you should check with modinfo if the module also supports these options.

maybe they are specific to mptscsi_2xx

Reply
0 Kudos
beovax
Enthusiast
Enthusiast
Jump to solution

esxcfg-module -l shows all the modules

modinfo megaraid2

# modinfo megaraid2

filename: /lib/modules/2.4.21-47.0.1.ELvmnix/kernel/drivers/scsi/megaraid2.o

description: "LSI Logic MegaRAID driver"

author: "LSI Logic Corporation"

license: "GPL"

parm: max_cmd_per_lun int, description "Maximum number of commands which can be issued to a single LUN (default=DEF_CMD_PER_LUN=63)"

parm: max_sectors_per_io short, description "Maximum number of sectors per I/O request (default=MAX_SECTORS_PER_IO=128)"

parm: max_mbox_busy_wait short, description "Maximum wait for mailbox in microseconds if busy (default=MBOX_BUSY_WAIT=10)"

Do you think its worth adjusting the max sectors per IO??

Reply
0 Kudos
whosmatt
Contributor
Contributor
Jump to solution

Hi,

i'm wondering how to fix this problem in ESXi 3.5; I have a Dell 2950 with Perc6 (LSI 1078) and SATA RAID5 and the performance is fine except for the stuttering. Are the paramaters that set something that I can set in the BIOS rather than at the driver level?

Thanks, and sorry to drag up an old topic like this but it seems like the problem

Reply
0 Kudos