SanDisk sx350-3200 not used as cache tier

[stephan87]

Hi,

we have a 8 node stretched cluster with dell 730xd (4 per datacenter) with witness on a third location. Config as followed per node

- 21 sas hdd

- 3 SanDisk sx350-3200

- add disk to storage: automatic

The SanDisk cards are on vsan hcl with driver version 4.2.3 but sandisk has revoked the driver because of a bug and now we have 4.2.4 installed (screenshot attached.) But the new driver is not on the hcl at the moment and the vsan health check shows them with a warning. Maybe is that the problem? When we run a i/o meter virtual machine the vsan observer shows at read cache 0.0 used (screenshot attached).

I have also run multicast, storage performance and vm creation test. Test results were ok with warning and storage test with poor performance. (screenshot attached)

VSAN health check reports the cache devices with warning. I have created some screenshots.

Regards,

Stephan

[zdickinson]

Good morning, was all running normally with the 4.2.3 driver and then these issues popped up after moving to the 4.2.4 driver?  I understand that SanDisk pulled the 4.2.3 driver, but unless you were having major issues, I would default to the HCL.  Thank you, Zach.

[pfuhli]

The 4.2.3 version of the driver is not available for download anymore. We started from scratch with 4.2.4.

[mschubi]

Hello,

there are some news. If we build a policy with 100% reservation we see cache usage.

Cache Usage is now

But after running IOMeter for a while the results are not so good as expected 😞

IOMeter runs with following metrics:

- 10GB Testsize

- 4 Workers

- 64 outstanding IOs

- 4K Block Size

- 70/30% read/write

- 80% random

- 30 Seconds Ramp Up

best regards,

Mike

[zdickinson]

Bummer, this does seem to pop up more than one would like.  Catch-22, the driver on the HCL is not available.  Have you contact SanDisk support to see about getting the driver?  Thank you, Zach.

[zdickinson]

I'm curious what you were expecting.  From one VM running IOMeter, that looks pretty good to me.  13k+ IOPS @ 4k is pretty good.  If you want to see more throughput, up the block size.

If you spin up IOMeter on several different VMs, can they all push 13k+ IOPS?  Thank you, Zach.

[mschubi]

Hello Zach,

there is nothing "good".

This single VM runs with 100% Readcache. Writecache is also more than enough available.

The same test VM give us more than 138K IO/s by <0.1s latency if we use the SX350 Flashdrives as VMFS

datastore.

We see only 13K IO/s by latencies between 5ms and 43ms.

Thats not an Flash based "Enterprise" performance...

best regards,

Mike

[stephan87 · Accepted solution]

We have applied the following parameters and it works and the heap message goes away:

esxcfg-advcfg -s 2047 /LSOM/heapSize

We have also applied two value for our controller from: https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=21354...

esxcfg-advcfg -s 110000 /LSOM/diskIoTimeout

esxcfg-advcfg -s 1 /LSOM/diskIoRetryFactor