Hi,
WARNING: Enabling WC without BBU will cause data corruption when system is shutoff incorrectly.
I have found quite a few posting regarding performance issues with ESXi and LSI IR. Probably due to the write caching being disabled by default on these controllers.
You can change this with LSIUtil (http://www.lsi.com/DistributionSystem/AssetDocument/support/downloads/hbas/fibre_channel/hardware_drivers/LSIUtil%20Kit_1.60.zip), but by default this does not work on ESXi because it's slightly different:
/tmp # ./lsiutil
LSI Logic MPT Configuration Utility, Version 1.60, July 11, 2008
sh: /sbin/modprobe: not found
sh: /bin/mknod: not found
Couldn't open /dev/mptctl or /dev/mpt2ctl!
0 MPT Ports found
/tmp #
I made the following change to the source of LSIUtil and then it works on my ESXi 4.0:
--- lsiutil/mpt2sas_ctl.h 2008-04-17 20:30:40.000000000 +0200
+++ lsiutil.fix/mpt2sas_ctl.h 2009-07-05 22:58:58.000000000 +0200
@@ -56,8 +56,8 @@
/**
HACK - changeme (MPT_MINOR = 220 )
*/
-#define MPT2SAS_MINOR MPT_MINOR+1
-#define MPT2SAS_DEV_NAME "mpt2ctl"
#define MPT2SAS_MINOR MPT_MINOR2
+#define MPT2SAS_DEV_NAME "mptctl_sas"
#define MPT2_MAGIC_NUMBER 'm'
#define MPT2_IOCTL_DEFAULT_TIMEOUT (10) /* in seconds */
-
/tmp # ./lsiutil.new
LSI Logic MPT Configuration Utility, Version 1.60, July 11, 2008
sh: /sbin/modprobe: not found
sh: /bin/mknod: not found
1 MPT Port found
Port Name Chip Vendor/Type/Rev MPT Rev Firmware Rev IOC
1. /proc/mpt/ioc0 LSI Logic SAS1068E B3 105 011c0200 0
1. Identify firmware, BIOS, and/or FCode
2. Download firmware (update the FLASH)
4. Download/erase BIOS and/or FCode (update the FLASH)
8. Scan for devices
10. Change IOC settings (interrupt coalescing)
13. Change SAS IO Unit settings
16. Display attached devices
20. Diagnostics
21. RAID actions
22. Reset bus
23. Reset target
42. Display operating system names for devices
45. Concatenate SAS firmware and NVDATA files
59. Dump PCI config space
60. Show non-default settings
61. Restore default settings
66. Show SAS discovery errors
69. Show board manufacturing information
97. Reset SAS link, HARD RESET
98. Reset SAS link
99. Reset port
e Enable expert mode in menus
p Enable paged mode
w Enable logging
Main menu, select an option: 21
1. Show volumes
2. Show physical disks
3. Get volume state
4. Wait for volume resync to complete
23. Replace physical disk
26. Disable drive firmware update mode
27. Enable drive firmware update mode
30. Create volume
31. Delete volume
32. Change volume settings
33. Change volume name
50. Create hot spare
51. Delete hot spare
99. Reset port
e Enable expert mode in menus
p Enable paged mode
w Enable logging
RAID actions menu, select an option: 32
Volume 0 is Bus 0 Target 0, Type IM (Integrated Mirroring)
Volume 0 Settings: write caching disabled, auto configure
Volume 0 draws from Hot Spare Pools: 0
Hot Spare Pools (bitmask of pool numbers):
RAID ACTION returned IOCLogInfo = 00010005
RAID actions menu, select an option: 0
Main menu, select an option: 0
-
I was doing a slow SCP while enabling WC and you can see my improvement in the attached screenshot.
Let me know if it works for you.
Does this change survive a reboot. If so which file is being changed ?? any ideas
chris
Yes, I have just tested and it survives a reboot.
The change is probably being made in the NVRAM of the LSI RAID controller, not anywhere on the VMware ESXi filesystem.
I can't get the lsiutil to run under ESXi 4.0.
I applied the patch you provided (thanks!) and compiled it via
gcc -Wall -static -O lsiutil.c -o lsiutil
When I try to run it on the ESXI host I only get
LSI Logic MPT Configuration Utility, Version 1.60, July 11, 2008
Segmentation fault
Thx
Heiko
I compiled it on i386.
If I compile it on x86-64, I get the same error as you.
On which architecture do you compile it?
I did first compile it on x86_64, but I also tried i386. I also tried it stripped and not-stripped.
This is what I currently got:
file lsiutil
lsiutil: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.0, statically linked, stripped
Maybe it's the glibc version, unfortunately I don't have an older one available.
Would you mind posting your working file here?
Heiko
I'd rather not post binaries of this. I'm not sure about the license of this software.
However, I do not build it statically, I just:
zoiah@matryoshka:~/tmp/Sourcefix/lsiutil.fix$ make
gcc -Wall -O lsiutil.c -o lsiutil
cp -p lsiutil /usr/bin
cp: cannot create regular file `/usr/bin/lsiutil': Permission denied
zoiah@matryoshka:~/tmp/Sourcefix/lsiutil.fix$ file lsiutil
lsiutil: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.8, not stripped
zoiah@matryoshka:~/tmp/Sourcefix/lsiutil.fix$
This is on a mostly vanilla Debian Lenny installation.
Please let me know if you can get it to work or not. If not, I'll see what I can do.
Okay I got it working.
It needs to be compiled for 32bit and as a dynamic executable. I was just thinking to much....
Thanks for the help!
Heiko
Can you confirm the same performance increase?
I'm copying my test VM over to the IR volume right now, it'll take a bit.
I'll post my findings as soon as I have them.
I'm seeing a HUGE difference in write performance!
The write speed is at least 2-3 times to what it was before.
I'm only testing in a Raid-1 right now, I hope I'll get a chance to try it on a Raid-1E this weekend.
Heiko
Hi,
Could I get a copy of your binary please?
Regards,
Darren
Hi,
Could I get a copy of your binary please?
Regards,
Darren
I created a blogpost about this subject (You cut the power to our ESXi with VMFS running on RAID with WriteCache on
without a BBU What happens next?) which also hosts the ESXi4 LSI Util I compiled.
You can find it here:
http://www.virtualistic.nl/archives/526
Hope you guys enjoy it (please visit some of the sponsors to keep the site alive)
Please consider awarding points if my response was helpful
Great bit of work this.
Assuming the controllers in question have no battery or flash cache backup though I would be sure to be running ESXi build 208xxx since LSI controller caches are not properly flushed at shutdown on earlier builds. Also a machine in that case a UPS and a UPS shutdown mechanism would be required.
Thanks..!!!
The sleepless night was actually caused by a remark I read on techhead.co.uk which inspired me to test it out. (remark probably made by you I guess )
I'm currently running VMware ESXi 4.0.0 build-219382 and am also working on getting the UPS to work with ESXi (using the excellent blogpost from techhead)
Aleks
Please consider awarding points if my response was helpful
Hi guys,
I'm having similar performance issues with an LSI SAS2008 controller mpt2sas drivers. Is there such a utility for this device on ESXi 5.0.0?
Many thanks
i would also be interested in a lsiutil that runs on esxi 5.0
can someone post a link to a working binary?
no news for a esxi5 solution?
This utility works in ESXi 5.1. I just tested it and I went from 1-10MB/s a second to 60MB/s+ steady copying over my gigabit link.