High-Capacity VSAN 6 Nodes

[jwhitehv]

In looking at the current VSAN 6 HCL, there are no high capacity SATA drives. Actually, no SATA drives at all for VSAN 6. Just wondering how others are putting high-capacity VSAN nodes together?

Is this a data error? Or a problem with SATA and VSAN 6? Or just a backlog in the qualification lab?

Without 4TB or 6TB drive options, the possible density per node plummets. Standard 2U servers generally come in either 12 x 3.5" or 24 x 2.5" drives. 12 x 4TB drives in VSAN 5.5 configs would yield 48TB of raw storage. 12 x 6TB drives gets 72TB raw. At least the 4TB drives are available in NL-SAS configs from some vendors and OEMs, but not all. Anyone doing this at all? Dell FX2 could get a full 35 drives, but the 2TB 2.5" SAS drives aren't on the HCL, so no 70TB 2.5" configuration.

Also, there's no FusionIO products on the list.

Maybe they're just on there as OEM parts? Actually, I double-checked and there are no PCIe cards on the VSAN 6 HCL.

How are other people doing high-capacity nodes?

Is 12 x 6TB per 2U server even possible with parts off the HCL?

[zdickinson]

I think he was answering my question about small compute power with a large amount of storage.  And yes, I can see that if you used something like Nexenta to serve up large amounts of storage you would need a small amount of compute.

72 cores in a 3 node cluster seems small to me.  I guess you are correct, it depends on the environment.

Back to your initial question of whether or not anyone had used vSAN at high density...  I don't have a direct answer, but Duncan has a had a small series of large scale deployments of vSAN.  Not sure of the density.  Thank you, Zach.

http://www.yellow-bricks.com/2015/07/06/virtual-san-is-breaking-down-silos-for-united-utilities/

Virtual SAN enabling PeaSoup to simplify cloud

[jwhitehv]

This is kinda nit picky. You would have the only listed options of HGST or OEM HP. That isn't to say your cluster wouldn't be supported by vmware, if you went with the Dell edition. I think the issue comes down to a frequent mantra of limited HCL testing capabilities, or something to that tune.

Wait, what? You're suggesting that VMware support is obligated to fully support systems not on the HCL?

[jwhitehv]

In regards to VMWare not supporting you when you aren't using HCL parts, I have not found this to be the case. I had 2 separate tickets open, support checked if everything was on HCL and warned me that data loss could occur from not following HCL recommendations and asked if I understood. Then proceeded to assist with my tickets.

That's helpful to know from a customer point of view. I'm approaching from a consultant's point of view, following the various design whitepapers from VMware which all hammer the HCL. On a conceptual basis, I can understand that the underlying point might be to use enterprise grade SSDs, controllers with stable drivers and a certain minimum queue depth, etc. But that's not what's happening. Effectively, the VSAN HCL is a list of parts that partners and consultants are limited to recommending from. A customer might make a decision to allow a deviation from it, but I can't imagine making a recommendation of parts not on the HCL, knowing that VMware is only going to give "best effort" support.

[elerium]

I absolutely agree, if I were consulting for customers, I'd stick to VSAN HCL only as that is stressed very much by VMWare as best practice there is risk from deviating from it. The current HCL does greatly limit in terms of price/capacity (no large SATA) and performance (no NVME drives).

It does look like they are slowly updating HCL, the WD RE4 4TB SAS was originally not in HCL when I implemented, I checked today and it's been qualified! :smileygrin:

[dlboone]

Not sure what the discrepancy is.  I'm seeing both high capacity drives and PCI-e SSDs for 6.0.  The Fusion-IOs show up under SanDisk.

[Bleeder]

Not sure what the discrepancy is.  I'm seeing both high capacity drives and PCI-e SSDs for 6.0.  The Fusion-IOs show up under SanDisk.

Hmm, did those get HCL'd for 6.0 recently?  I remember checking the SX300-1600 earlier in the month, and it was only HCL'd for 5.5.

[jonretting]

No -- I'm suggesting they are not necessarily mutually inclusive. In my opinion. Moreover that the HCL is under serviced. Thanks, -Jon

[jwhitehv]

No -- I'm suggesting they are not necessarily mutually inclusive. In my opinion. Moreover that the HCL is under serviced.

Well, I agree that product seems to be slow getting to the HCL. But again, from a consultant or partner point of view, it's difficult to recommend product not on the HCL hoping to get support instead of with product on it, knowing the customer will get support.

[jwhitehv]

Not sure what the discrepancy is.  I'm seeing both high capacity drives and PCI-e SSDs for 6.0.  The Fusion-IOs show up under SanDisk.

Yes, there are now PCIe drives certified for VSAN 6! These have literally appeared today! The Intel 3700 family on the scaling guide is still not there though.

Model	Device Type	Part Number	Capacity
Fusion ioMemory PX600-1000	PCI-E	SDFACAMOP-1T00-SF1	1000
Fusion ioMemory PX600-1300	PCI-E	SDFACAMOP-1T30-SF1	1300
Fusion ioMemory PX600-2600	PCI-E	SDFACAMOP-2T60-SF1	2600
Fusion ioMemory PX600-5200	PCI-E	SDFACCMOP-5T20-SF1	5200
Fusion ioMemory SX300-1300	PCI-E	SDFACAMOS-1T30-SF1	1250
Fusion ioMemory SX300-1600	PCI-E	SDFACAMOS-1T60-SF1	1600
Fusion ioMemory SX300-3200	PCI-E	SDFACAMOS-3T20-SF1	3200
Fusion ioMemory SX300-6400	PCI-E	SDFACAMOS-6T40-SF1	6400

[Bleeder]

Well, better late than never?

I'm a bit disappointed that there isn't a better way to be notified of updates to the VSAN HCL.  It looks like I'll have to check for updates manually.  I had been relying on RSS updates to the Virtual Blocks blog.

[jonretting]

Yup checking sites for updates is annoying, and even worse when they change something element/structurally. Typically if i have to check something on the web multiple times in a couple days, it is worth automating the process with a little bit shell/curl/smtp/git/cron. That all goes to shit however when i get an email containing "<div clas"  differs from the previous attempt. It seems the problem is lack of any standardization across any company/downloads. Typically you have to fumble along their ill conceived frontends, and gets even more aggravating when you have to asynchronously post form data. For the first time this year google api's was unreachable in SOCAL, even workarounds were hard due to SSL. A lot more time is usually spent just locating the item you know you need, have already worked out the fix, but for some blast-famous reason downloads.company.com is down. I would hate to be the sysadmin taking down the public downloads, since it's something that already eats away at my soul. Hehe Cheers, -Jon

[jwhitehv]

I've been writing up my experiences in this blog series: High Capacity VSAN Nodes