2-node VSAN with local witness appliance setup and config help?

[Pat0182]

Good morning all,

This is my first build of a VSAN and I have the following sitting on my desk...

2 x HP DL380G9's running an Intel DCS3710 + 3 x 4TB HDD (all SATA) off of a P840 and 192GB RAM.

1 x ESXi compatible OEM build whitebox and driver supported ESXi host with 3 x 4TB HDD running off of an LSI9260 with 32GB RAM.

The goal is to setup a little lab with the 2 DL380 G9's as the workhorses and hosts of course, and the whitebox as the vcenter server appliance + a nested VSAN witness appliance (or physical vsan witness if need be).

The idea is to be able to enable HA on the VM's running on the 2 x DL380 G9's.

May main reason for posting are to ask the awesome members of the community here a few questions.

1) please let me know of any potential hiccups I may expect to see if anything of my initial design jumps out at you.

2) I really don't understand whether I need the vsan advanced or standard license as what I am trying to do is a 2(3) node cluster which will all be locally housed in the same rack, not the suggested stretched cluster ROBO config.

other things that come to mind are:

- will I be able to get away with L2, not L3 routing as I am not technically hosting the witness appliance in a remote branch office?

- if the config for a 2-node cluster like I have here would work with a vsphere essentials plus kit + required vsan licenses...

Thank you all in advance and I look forward to hearing any advice/discussion that ensues...

[CrashTheGooner]

Dear Pat

Kindly note that VSAN requires minimum of one SSD in each of your HP servers
Also you will only be able to TURN ON vSAN if you have license for it .
You can apply any eval license like partner licenses etc should work fine.

Let us know what happens

[zdickinson]

Good morning, great questions.

I don't see any immediate hiccups.  SSDs look good.  Controller looks good.  They are SATA so I would test performance during a rebuild.

I think standard is fine.  You would need the advanced license if you wanted to do All Flash.

If you have everything on the same subnet, you should not need any L3 routing.

vSphere Essentials Plus and a Standard vSAN license should get you where you want to be.

Good luck and keep us updated.  Thank you, Zach.

[Pat0182]

Thanks for the heads up and vote of confidence guys!

I have configured VSAN on the above mentioned hardware and am pleasantly surprised by its genius. It was not as hard as I expected it to be considering hundreds of pages of documentation and hardware requirements to sift through.

I have a few more questions if you do not mind answering them...

1) I have not exactly allocated the recommended 1:10 flash to magnetic storage ratio - how can I expect this to impact performance in future? Especially if I go adding another 6 x 4TB magnetic drives with my 800GB SSD?

2) What is the most effective way for me to benchtest and get performance metrics regarding the VSANs performance under stress?

3) Currently the OEM whitebox runs the VCSA and the VSAN Witness Appliance. Is this a normal configuration? where do people normally put the VCSA in a 3 node cluster like this?

4) the workloads this will be doing will be heavily pushed by consistent and unrelenting CCTV footage streams in (and sometimes out for playback)... what would be your recommendations for avoiding issues such as SATA queue depth, or any other bottlenecks you can see? The DL380's have 4x1Gbps NICs + a 2 Port 10GbE card in them for future use when this system expands. LACP may be employed if needed depending on budget availability for 10GbE Switches.

5) Can someone please explain to me or point me to a guide where the VSAN Witness components are explained in lay-mans terms? I fail to understand or see where/why the witness components come into play in the greater scheme of things and if the performance of the Witness affects the performance of the VSAN?

6) any suggestions for not breaking things when I test the HA?

More thanks in advance!

[zdickinson]

I'll try my best, and make sure you double check everything

1) I have not exactly allocated the recommended 1:10 flash to magnetic storage ratio - how can I expect this to impact performance in future? Especially if I go adding another 6 x 4TB magnetic drives with my 800GB SSD?

Well, this is interesting actually.  If you add that much capacity tier and keep the default stripe width to 1, then you can expect performance to suffer.  Each write will hit SSD and then destage to 1 disk.  However, if you add capacity and increase the stripe width appropriately you could see no degraded performance.  If you increased stripe width enough, you may even see it increase.  Obviously more SSD the better.

2) What is the most effective way for me to benchtest and get performance metrics regarding the VSANs performance under stress?

I believe HCL Bench is a thing.  Have not used it.  I used IO Analyzer, a VMware fling.  I/O Analyzer – VMware Labs  It has several canned workloads.  I also just installed IO Meter on a few Windows VMs and crunched the storage.

3) Currently the OEM whitebox runs the VCSA and the VSAN Witness Appliance. Is this a normal configuration? where do people normally put the VCSA in a 3 node cluster like this?

Can't help with this one.

4) the workloads this will be doing will be heavily pushed by consistent and unrelenting CCTV footage streams in (and sometimes out for playback)... what would be your recommendations for avoiding issues such as SATA queue depth, or any other bottlenecks you can see? The DL380's have 4x1Gbps NICs + a 2 Port 10GbE card in them for future use when this system expands. LACP may be employed if needed depending on budget availability for 10GbE Switches.

With the SSD cache layer, I think, in general vSAN is better with burst work loads and you may see issues with constant IO.  However, this goes back to number 1, increasing the number of capacity disks and stripe width will help.

5) Can someone please explain to me or point me to a guide where the VSAN Witness components are explained in lay-mans terms? I fail to understand or see where/why the witness components come into play in the greater scheme of things and if the performance of the Witness affects the performance of the VSAN?

I think this article from Cormac might do the trick:  VSAN 6.0 Part 1 - New quorum mechanism - CormacHogan.com  There is a new mechanism to reach quorum in vSAN 6.

6) any suggestions for not breaking things when I test the HA?

Don't stop until you break it.  We have a three node cluster, so FTT = 1.  I pulled power on one.  All good.  Then I pulled power on another, FTT not met = everything down.  Then pulled power on the last and brought them up in reverse order.  Break it so you know what not to do.

Thank you, Zach.