Reply to Message

View discussion in a popup

Replying to:
logan892
Contributor
Contributor

LSI SAS 2008 RAID 1 always thrashing

I'm attempting to use an LSI 9212-4i4e in IR mode with ESXi 6.0. I have configured two Western Digital RE 320 GB drives in RAID 1.

Firmware on the controller is P20, and I also loaded the P20 drives from the website.

There is no on-board cache on this controller.

Drive activity lights were blinking while the drives were initializing, and of course write performance was horrendous.

Writing anything caused the drives to thrash, and sequential writing speeds (testing with dd) reached no more than 2 MB/s.

After several hours (drives left to their initialization over night) the initialization appears to have finished and the drive activity has quieted down. Drive activity lights are no longer constantly blinking, and remain mostly quiet unless I initiate a write myself.

However, the sequential write performance is still absolutely terrible. Write speed has only doubled compared with when the drives were initializing, reaching approximately 4-5 MB/s.

The drives are still thrashing constantly when writing to the volume.

I've tried to remove and recreate the vmfs volume. The result is the same.

Writing 150 MB (from /dev/zero) takes 30+ seconds. Reading back the same 150 MB takes 4 seconds.

Writes: 4.5 MB/s

Writes: 37 MB/s

[root@esxi6:~] date; time dd if=/dev/zero of=/vmfs/volumes/vmstore2_raid1_wd_re/out.tmp bs=1048576 count=150

Tue Mar 22 10:04:45 UTC 2016

150+0 records in

150+0 records out

real    0m 33.20s

user    0m 0.08s

sys     0m 0.00s

[root@esxi6:~] date; time dd of=/dev/null if=/vmfs/volumes/vmstore2_raid1_wd_re/out.tmp bs=1048576 count=150

Tue Mar 22 10:06:23 UTC 2016

150+0 records in

150+0 records out

real    0m 4.05s

user    0m 0.10s

sys     0m 0.00s

[root@esxi6:~] date; time dd if=/dev/zero of=/vmfs/volumes/vmstore2_raid1_wd

_re/out.tmp bs=1048576 count=300

Tue Mar 22 10:07:35 UTC 2016

300+0 records in

300+0 records out

real    1m 12.24s

user    0m 0.15s

sys     0m 0.00s

[root@esxi6:~] date; time dd of=/dev/null if=/vmfs/volumes/vmstore2_raid1_wd

_re/out.tmp bs=1048576 count=300

Tue Mar 22 10:08:55 UTC 2016

300+0 records in

300+0 records out

real    0m 7.63s

user    0m 0.22s

sys     0m 0.00s

cmd: esxcli storage core device list

naa.600508e00000000071e6339d7559b40d

   Display Name: LSI Serial Attached SCSI Disk (naa.600508e00000000071e6339d7559b40d)

   Has Settable Display Name: true

   Size: 304222

   Device Type: Direct-Access

   Multipath Plugin: NMP

   Devfs Path: /vmfs/devices/disks/naa.600508e00000000071e6339d7559b40d

   Vendor: LSI

   Model: Logical Volume

   Revision: 3000

   SCSI Level: 6

   Is Pseudo: false

   Status: degraded

   Is RDM Capable: true

   Is Local: false

   Is Removable: false

   Is SSD: false

   Is VVOL PE: false

   Is Offline: false

   Is Perennially Reserved: false

   Queue Full Sample Size: 0

   Queue Full Threshold: 0

   Thin Provisioning Status: unknown

   Attached Filters:

   VAAI Status: unknown

   Other UIDs: vml.0200000000600508e00000000071e6339d7559b40d4c6f67696361

   Is Shared Clusterwide: true

   Is Local SAS Device: false

   Is SAS: true

   Is USB: false

   Is Boot USB Device: false

   Is Boot Device: false

   Device Max Queue Depth: 128

   No of outstanding IOs with competing worlds: 32

   Drive Type: logical

   RAID Level: RAID1

   Number of Physical Drives: 2

   Protection Enabled: false

   PI Activated: false

   PI Type: 0

   PI Protection Mask: NO PROTECTION

   Supported Guard Types: NO GUARD SUPPORT

   DIX Enabled: false

   DIX Guard Type: NO GUARD SUPPORT

   Emulated DIX/DIF Enabled: false

[root@esxi6:~] esxcli storage core device raid list -d naa.600508e0000000007

1e6339d7559b40d

   Physical Location: enclosure 0, slot 6

   Physical Location: enclosure 0, slot 7

cmd: esxcli storage core device vaai status get

naa.600508e00000000071e6339d7559b40d

   VAAI Plugin Name:

   ATS Status: unsupported

   Clone Status: unsupported

   Zero Status: supported

   Delete Status: unsupported

[root@esxi6:~] esxcli storage core device partition list

Device                                                                      Partition  Start Sector  End Sector  Type           Size

--------------------------------------------------------------------------  ---------  ------------  ----------  ----  -------------

t10.ATA_____WDC_WD1003FBYX2D50Y7B1________________________WD2DWCAW36KXSCNY          0             0  1953525168     0  1000204886016

t10.ATA_____WDC_WD1003FBYX2D50Y7B1________________________WD2DWCAW36KXSCNY          1          2048  1953523712    fb  1000203091968

mpx.vmhba33:C0:T0:L0                                                                0             0    61457664     0    31466323968

mpx.vmhba33:C0:T0:L0                                                                1            64        8192     0        4161536

mpx.vmhba33:C0:T0:L0                                                                5          8224      520192     6      262127616

mpx.vmhba33:C0:T0:L0                                                                6        520224     1032192     6      262127616

mpx.vmhba33:C0:T0:L0                                                                7       1032224     1257472    fc      115326976

mpx.vmhba33:C0:T0:L0                                                                8       1257504     1843200     6      299876352

mpx.vmhba33:C0:T0:L0                                                                9       1843200     7086080    fc     2684354560

t10.ATA_____Samsung_SSD_850_PRO_256GB_______________S1SUNSAG346333H_____            0             0   500118192     0   256060514304

t10.ATA_____Samsung_SSD_850_PRO_256GB_______________S1SUNSAG346333H_____            1          2048   419432448    fb   214748364800

naa.600508e00000000071e6339d7559b40d                                                0             0   623046656     0   318999887872

naa.600508e00000000071e6339d7559b40d                                                1          2048   623046623    fb   318998822400

My main VM store is currently on a single Samsung SSD, connected to the internal Intel controller. While performance is not stellar here either, it is able to reach above 100 MB/s writes which is much more acceptable.

What may be causing this constant HDD seek/thrashing while writing to the RAID 1 array, and is there a way to mitigate it in ESXi 6.0?

Should I try downgrading the firmware to P19 and also reverting the VMware drivers to P19?

Can I activate or deactivate some VMFS related features?

Can I from ESXi modify the drive cache behavior?

Am I s**t-out-of-luck with seeing any decent write performance using this controller, while retaining some semblance of data integrity in the event of power failure, for example?

Reply
0 Kudos