Skip navigation
VMware
2,423 Views 17 Replies Last post: Feb 8, 2012 5:20 AM by Scribe RSS
1 2 Previous Next
Smite Lurker 5 posts since
Jul 5, 2009
Currently Being Moderated

Jul 5, 2009 2:22 PM

LSI IR writecaching performance fix

Hi,

 

WARNING: Enabling WC without BBU will cause data corruption when system is shutoff incorrectly.

 

I have found quite a few posting regarding performance issues with ESXi and LSI IR. Probably due to the write caching being disabled by default on these controllers.

 

You can change this with LSIUtil (http://www.lsi.com/DistributionSystem/AssetDocument/support/downloads/hbas/fibre_channel/hardware_drivers/LSIUtil%20Kit_1.60.zip), but by default this does not work on ESXi because it's slightly different:

 

/tmp # ./lsiutil

 

LSI Logic MPT Configuration Utility, Version 1.60, July 11, 2008

sh: /sbin/modprobe: not found

sh: /bin/mknod: not found

Couldn't open /dev/mptctl or /dev/mpt2ctl!

 

0 MPT Ports found

/tmp #

 

I made the following change to the source of LSIUtil and then it works on my ESXi 4.0:

 

--- lsiutil/mpt2sas_ctl.h       2008-04-17 20:30:40.000000000 +0200

+++ lsiutil.fix/mpt2sas_ctl.h   2009-07-05 22:58:58.000000000 +0200

@@ -56,8 +56,8 @@

/**

   

  • HACK - changeme (MPT_MINOR = 220 )

  */

-#define MPT2SAS_MINOR          MPT_MINOR+1

-#define MPT2SAS_DEV_NAME       "mpt2ctl"

#define MPT2SAS_MINOR          MPT_MINOR2

+#define MPT2SAS_DEV_NAME       "mptctl_sas"

#define MPT2_MAGIC_NUMBER      'm'

#define MPT2_IOCTL_DEFAULT_TIMEOUT (10) /* in seconds */

 

-


/tmp # ./lsiutil.new

 

LSI Logic MPT Configuration Utility, Version 1.60, July 11, 2008

sh: /sbin/modprobe: not found

sh: /bin/mknod: not found

 

1 MPT Port found

 

     Port Name         Chip Vendor/Type/Rev    MPT Rev  Firmware Rev  IOC

1.  /proc/mpt/ioc0    LSI Logic SAS1068E B3     105      011c0200     0

 

Select a device:  LSI IR writecaching performance fix 1

 

1.  Identify firmware, BIOS, and/or FCode

2.  Download firmware (update the FLASH)

4.  Download/erase BIOS and/or FCode (update the FLASH)

8.  Scan for devices

10.  Change IOC settings (interrupt coalescing)

13.  Change SAS IO Unit settings

16.  Display attached devices

20.  Diagnostics

21.  RAID actions

22.  Reset bus

23.  Reset target

42.  Display operating system names for devices

45.  Concatenate SAS firmware and NVDATA files

59.  Dump PCI config space

60.  Show non-default settings

61.  Restore default settings

66.  Show SAS discovery errors

69.  Show board manufacturing information

97.  Reset SAS link, HARD RESET

98.  Reset SAS link

99.  Reset port

e   Enable expert mode in menus

p   Enable paged mode

w   Enable logging

 

Main menu, select an option:  LSI IR writecaching performance fix 21

 

1.  Show volumes

2.  Show physical disks

3.  Get volume state

4.  Wait for volume resync to complete

23.  Replace physical disk

26.  Disable drive firmware update mode

27.  Enable drive firmware update mode

30.  Create volume

31.  Delete volume

32.  Change volume settings

33.  Change volume name

50.  Create hot spare

51.  Delete hot spare

99.  Reset port

e   Enable expert mode in menus

p   Enable paged mode

w   Enable logging

 

RAID actions menu, select an option:  LSI IR writecaching performance fix 32

 

Volume 0 is Bus 0 Target 0, Type IM (Integrated Mirroring)

 

Volume 0 Settings:  write caching disabled, auto configure

Volume 0 draws from Hot Spare Pools:  0

 

Enable write caching:  LSI IR writecaching performance fix yes

Offline on SMART data:  LSI IR writecaching performance fix

Auto configuration:  LSI IR writecaching performance fix

Priority resync:  LSI IR writecaching performance fix

Hot Spare Pools (bitmask of pool numbers):  LSI IR writecaching performance fix

 

RAID ACTION returned IOCLogInfo = 00010005

 

RAID actions menu, select an option:  LSI IR writecaching performance fix 0

 

Main menu, select an option:  LSI IR writecaching performance fix 0

-


 

I was doing a slow SCP while enabling WC and you can see my improvement in the attached screenshot.

 

Let me know if it works for you.

Attachments:
ceemour Enthusiast 80 posts since
Apr 7, 2006
Currently Being Moderated
1. Jul 5, 2009 3:20 PM in response to: Smite
Re: LSI IR writecaching performance fix

 

Does this change survive a reboot. If so which file is being changed ?? any ideas

 

 

chris

 

 

hzuerker Novice 7 posts since
Feb 7, 2007
Currently Being Moderated
3. Jul 10, 2009 6:21 AM in response to: Smite
Re: LSI IR writecaching performance fix

I can't get the lsiutil to run under ESXi 4.0.

 

I applied the patch you provided (thanks!) and compiled it  via

 

gcc -Wall -static -O lsiutil.c -o lsiutil

 

When I try to run it on the ESXI host I only get

 

LSI Logic MPT Configuration Utility, Version 1.60, July 11, 2008

Segmentation fault

 

Thx

  Heiko

hzuerker Novice 7 posts since
Feb 7, 2007
Currently Being Moderated
5. Jul 10, 2009 6:41 AM in response to: Smite
Re: LSI IR writecaching performance fix

I did first compile it on x86_64, but I also tried i386. I also tried it stripped and not-stripped.

 

This is what I currently got:

file lsiutil

lsiutil: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.0, statically linked, stripped

 

Maybe it's the glibc version, unfortunately I don't have an older one available.

 

Would you mind posting your working file here?

 

Heiko

hzuerker Novice 7 posts since
Feb 7, 2007
Currently Being Moderated
7. Jul 10, 2009 6:53 AM in response to: Smite
Re: LSI IR writecaching performance fix

Okay I got it working.

It needs to be compiled for 32bit and as a dynamic executable. I was just thinking to much....

 

Thanks for the help!

 

Heiko

hzuerker Novice 7 posts since
Feb 7, 2007
Currently Being Moderated
9. Jul 10, 2009 7:03 AM in response to: Smite
Re: LSI IR writecaching performance fix

I'm copying my test VM over to the IR volume right now, it'll take a bit.

I'll post my findings as soon as I have them.

hzuerker Novice 7 posts since
Feb 7, 2007
Currently Being Moderated
10. Jul 10, 2009 9:44 AM in response to: hzuerker
Re: LSI IR writecaching performance fix

I'm seeing a HUGE difference in write performance!

The write speed is at least 2-3 times to what it was before.

I'm only testing in a Raid-1 right now, I hope I'll get a chance to try it on a Raid-1E this weekend.

 

Heiko

DAPBENJAMIN Novice 30 posts since
Jan 29, 2009
Currently Being Moderated
11. Jan 7, 2010 8:25 AM in response to: Smite
Re: LSI IR writecaching performance fix

Hi,

 

Could I get a copy of your binary please?

 

Regards,

 

Darren

DAPBENJAMIN Novice 30 posts since
Jan 29, 2009
Currently Being Moderated
12. Jan 7, 2010 8:25 AM in response to: hzuerker
Re: LSI IR writecaching performance fix

Hi,

 

Could I get a copy of your binary please?

 

Regards,

 

Darren

alphenit Enthusiast 83 posts since
Feb 9, 2006
Currently Being Moderated
13. Mar 5, 2010 3:05 AM in response to: DAPBENJAMIN
Re: LSI IR writecaching performance fix

I created a blogpost about this subject (You cut the power to our ESXi with VMFS running on RAID with WriteCache on

without a BBU What happens next?) which also hosts the ESXi4 LSI Util I compiled.

You can find it here:

http://www.virtualistic.nl/archives/526

 

Hope you guys enjoy it (please visit some of the sponsors to keep the site alive)

 

 

 

 

Please consider awarding points if my response was helpful

*Please consider awarding points if my response was helpful*
J1mbo Virtuoso 1,815 posts since
May 20, 2009
Currently Being Moderated
14. Mar 5, 2010 3:14 AM in response to: alphenit
Re: LSI IR writecaching performance fix

Great bit of work this.

 

Assuming the controllers in question have no battery or flash cache backup though I would be sure to be running ESXi build 208xxx since LSI controller caches are not properly flushed at shutdown on earlier builds.  Also a machine in that case a UPS and a UPS shutdown mechanism would be required.

blog.peacon.co.uk

Bookmarked By (0)

Share This Page

Communities