Vmware esxi 3.5 with p400 and poor performance

[cefaucz]

Hello, I have the following problem. I use VmWare ESXi server 3.5 update 2 and have performance problems.

Hardware:

HP ML370 G5

CPU 2x 2,66 Ghz QuadCore

RAM 32 GB

Raid Controller HP P400 SAS

RAID5 with 4 x 146GB

Guest:

SLES 10 SP2 / Samba Server

Problem:

I did a copy from my samba server to my samba server by ssh console and performance was very bad. The copy is nothing more than 10 MBytes/sec.

I installed a Windows 2003 SE directly on my machine and the performance to write on disk was 5x better than vmware exsi 3.5 with guest linux.

I would like to know if there any update this raid controller to vmware exsi 3.5 installable.

Thanks in advance for your assistance

[Dave_Mishchenko]

Welcome to the VMware Community Forums. Your post has been moved to the Performance forum.

Dave Mishchenko

VMware Communities User Moderator

[Enimus]

Hi there,

While my hardware is different than yours, I have exactly the same problem. Let me ask the obvious question first.

Does your Raid controller have a BBU (battery backup unit)?

[cefaucz]

Hi,

My raid controller doesn't have a BBWC. I already anothers posts that saw without BBWC the performance is poor.

However, I don't understand how my windows host write on disk at around 70 MB/s and my vmware exsi 3.5 host write on disk at around 10 MB/s.

Thanks,

Carlos.

[Enimus]

From what I have seen, its seems that ESXi will notice the lack of BBWC, and disbale it accordingly.

Hence the crappy performance. Get a BBWC, and try again.

[cefaucz]

Is there a workaround/setup to enable controller cache works same without BBWC?

[Ken_Cline]

The BBWC allows the system to perform "lazy writes" - this means that the controller will store the data that needs to be written to disk in its local cache and acknowledge back to the OS (in this case, ESX) that the data has been saved. At a later time, when there's nothing else happening, the controller can then flush its cache to the actual hard drive. This cannot be done without the battery backup, because if the system failed before the controller could flush its cache, you would have a corrupted filesystem (NOT a good thing ).

Without the BBWC the controller does not acknowledge the write until the data has been committed to magnetic media (i.e. the disk drive). This means that you now have rotating media injected into the write process, resulting in significantly longer acknowledgement times for each write operation.

Short answer - get a battery backed write-through cache enabled controller!

Ken Cline

Technical Director, Virtualization

Wells Landers

TVAR Solutions, A Wells Landers Group Company

VMware Communities User Moderator

[Con149]

Hello,

we have a p400/256 with Battery Status OK 25/75% Read Write Cache, but the Same Problem. The Raid 1 is fast with 50MB/s an d the Raid 5 very slow with 5MB/s.

Firmware 5.20 / ESX3.5U2 with all Patches.

[Navid_Baradaran]

Hi everybody,

I have two server with identical P400/512 + BBWC, one HP DL380G5 and another DL580G5,

and have Windows Server 2003 as guests of ESXi 3.5

problem is a very poor performance around 5-6 MB/s

I have 20 MB/s in my network so my clients are much more faster than my server !!!

this is affecting my database performance and file copy long delays.

I am using RAID 6 (ADG) on 8 SAS SFF 10K disks.

any idea?

thanks,

Navid

[Rich-Ontai]

then what your seeing is an issue with copying or moving files from the vmfs partition through the service console, not disk performance issues.

For some reason, i dont' know why, but copying or moving files from vmfs through the service console is horrible on all ESX servers. The answer is to use FastSCP from Veeam. It will decrease your copy times by a factor of 16. A 10G VM will take about 2.5 hours on most ESX servers using CLI scp. With FastSCP, you can do the copy in 15 minutes.

LIke i said, i dont' know why it usually takes so long to copy files off the vmfs partition, but FastSCP is your answser. There's no agents or install to do on the ESX host, just a simple install on a windows box and your off to the races. And FastSCP is free.

I apologize in advance if i've misread your post..

Do you see " horrible performance" from another source that copying off the vmfs partition?

[Navid_Baradaran]

Thank you for very useful information, I am now geting FastSCP but actually that was not my exact issue. by the way, I am using ESXi and I think that FastSCP only supports ESX not ESXi. isn' it?

I am not trying to vopy ISO VM or etc. from ESX to ESX or ESX to guest, I have a very poor performance inside my guest os. even internal file copy of a guest server have the maximum of 7 MBps, its killing.

after some investigations I found that my P400/512 write cache Battery is FAILED!! and this may force the system to slowing down. I am currently replacing the BBWC and I hope that this solve my problem.

Navid

[Rich-Ontai]

yes, the write back needs to be enabled, according to other posts here and from personal experience.

Veeam will be supporting ESXi soon , from what i've read. I've also read that if you add Vitrual Center to the list of servers, instead of doing the ESX server you can use FastSCP or Veeam Backup.

Sorry for the confusion.

[georgemason]

Hi,

I've installed a DL380 G5 with ESXi 3.5 and am experiencing similar issues with disk write performance. It's connected to the network by single gigabit NIC which at the moment is the management and Vm nic, but can't see that this is causing a bottleneck (currently the server is under no load at all) as copying between volumes within a VM gives the same result.

Can anyone shed any light on how I check whether the BBWC is enabled? In Windows I'd run the Systems Mgmt Homepage but obviously I need to do something different in this instance.

Any help appreciated. Thanks.

George

[philipharvey]

I have the same issue on DL380G5 with P400 raid controller. RAID5 performance is about 10MB/sec, which is about 1/10th of the speed it should be.

A post over here: http://communities.vmware.com/message/583387

says that the problem is that VMWare never lets the array controller finish "building the array".

I assume this is what HP refers to as the peformance optomization. The post is a little short on details but I will try to replicate the problem in my lab and see if I can document the solution.

[phnz]

Looks like this problem with the shitty HP P400 raid controller isn't just on VMWare, check out this thread.

http://forums13.itrc.hp.com/service/forums/questionanswer.do?admit=109447627124575091112928353475&threadId=1240003

[phnz]

I opened an incident with HP Support, and their reply was that it is a known fault with the P400 that RAID5 write is crappy on this controller without a battery and I should buy a battery.

They also referred me to this page on the HP web site.

http://h10025.www1.hp.com/ewfrf/wc/document?docname=c01670205&cc=my&dlc=en&lc=en&jumpid=reg_R1002_MY...

[georgemason]

I think the long and short is that ESX + P400 - BBWC = crappy performance. I spec BBWC for every ESX box I install on HP hardware these days.