VMware Cloud Community
m0ps
Enthusiast
Enthusiast

P3700 with less capacity vs P3600 with more capacity as vSAN flash cache device?

Hi All!

I have a small question about vSAN flash cache device.

When implementing vSAN datastore what choose is preferable:

1) Small SSD, but with more endurance and performance

2) Large SSD, but with less  endurance and performance

?

For example we have 3-node cluster with 7*2TB NL-SAS. As our budget, for flash cache we can acquire P3700 800Gb or P3600 1600Gb.

Another one sided question - is it applicable mix nodes with not equal storage devices (HDDs)? For example 3 nodes with 1SSD+7HDDs, and fourth with 1SSD+3HDDs?

best regards, m0ps
Reply
0 Kudos
6 Replies
zdickinson
Expert
Expert

Good morning, I would follow the 10% rule.  Your SSD should be 10% of your working data set.  Note that's not 10% of total VMDK, but 10% of your data that is hot.  If the smaller one can meet that, do that.  If not, use the larger one.

You can mix configs as you outlined, but it's not recommended as it may cause inconsistent performance.  Thank you, Zach.

gmtx
Hot Shot
Hot Shot

I had the same question, and it really depends if you're only asking about endurance or also performance.

If it's endurance, then according to the VMware guys working with me on my configuration the maximum supported usable size for the cache layer is currently 800GB, meaning your 1.6TB drive is only ever going to get half used. Adding a cache disk larger than 800GB won't get you any additional performance, but in theory should help with endurance due to wear leveling effectively using the full internal SSD capacity. (But you can't be certain - seems wear leveling is proprietary black magic where details are rarely disclosed.)

Reply
0 Kudos
elerium
Hot Shot
Hot Shot

I don't believe this is actually correct. On all flash configs, the cache layer only uses 600GB max and is used only for writes, going above 600GB for all flash cache may not make much sense other than for endurance reasons. The OP is using a hybrid config so this wouldn't apply.

On one of my hybrid configs I have 2TB P3700 for SSD cache. The entirety of the drive is being used for caching. For hybrid config SSD cache is used in a 70/30 (read/write) ratio. I had the same question about P3700 vs P3600, I have clusters that utilize both now so I can tell you some differences other than cost. Use P3700 if you really need the endurance or better write IOPS performance otherwise I have generally found P3600 works well and is very good price/performance for most applications since most things I have are read heavy. The P3700 is significantly faster with write performance than P3600, it is a noticable performance increase with heavy write intensive database apps.

In the attached picture:

Total cache capacity: 1.82TB

Read cache size: 1.27TB

Used capacity: 1.19TB

Write buffer size: 558.9GB

Reply
0 Kudos
m0ps
Enthusiast
Enthusiast

So... who is right? %)

best regards, m0ps
Reply
0 Kudos
zdickinson
Expert
Expert

Well, I can clearly see the picture; but the documentation I have seen supports the max of 800 GB and any more than that only helps with endurance.  However a quick search through the newest Duncan-Cormac book, doesn't mention it.  Perhaps that was the case in older versions.  Thank you, Zach.

Reply
0 Kudos
JohnNicholsonVM
Enthusiast
Enthusiast

Its an all flash limit, but the thing I would like to point out is that deploying larger NL-SAS drives has issues beyond just read cache miss performance.

The P3700 will have better write latency constancy is one benefit over the P3600 (not that its a slouch). 

If you put start stacking (4TB) NL-SAS drives in hosts, you can end up in a position where it takes a week to evacuate a host.  Compression and Dedupe, combined with lower costs on flash devices (Dell's website is showing ~54 cents per GB right now for me for the Samsung TLC drives) and RAID 5/6 protection levels means that All flash is cheaper than 15K, 10K, and in some cases potentially "close enough" to NL-SAS drives. If its a greenfield deployment I would strongly consider all flash.

Reply
0 Kudos