Creating new vsan - will this work??

[kaschenberg]

Hi,

Let me first start off with my environment. Running vcenter 6.7 and esx hosts all on 6.7 (everything is on latest version to this day). I have 5 esx hosts joined to a cluster with shared iscsi storage. 2 of these hosts have 24 x 900gb drives and each with 1 x 800gb ssd. I figured, why not create a vsan with these unused drives, right?!

So, I walk through the vsan configuration to enable it, claim disks on both hosts for capacity and caching tier and put host A in 1 fault domain and host B in 1 fault domain to separate them. Now, I'm at the "Finish" step and before I proceed and click the button, does anyone know if this will work?

I was told that with a 2 node cluster, a witness appliance is needed, but the vsan config wizard seems to let me get around this?

Also, being that this is a production environment, will anything go "down" on these hosts when the vsan configuration is applied?

And one last question, can a 2 node vsan configuration reside in a cluster with 5 hosts in it?

Thanks

[sk84]

In short, don't do it.

First you should check if your disks and the raid controllers are on the vSAN HCL and supported and also have the supported firmware and drivers installed. This is essential.

VMware Compatibility Guide - vsan

For a 2 node cluster there has to be a witness appliance or witness host. However, you don't have a 2 node cluster, because vSAN always works cluster-wide. So you have a 5 node cluster, of which only 2 nodes are configured correctly and only these two nodes will contribute disk capacity. Generally it is supported that not all hosts in a cluster contribute storage to vSAN, but it's not recommendable. But all hosts in a cluster should be configured for vSAN anyway.

Another thing are the disk groups. A disk group can contain a maximum of 7 capacity disks and each disk group needs one cache device in a hybrid setup. If your servers have only 1 SSD, you can only create 1 disk group with a maximum of 7 disk capacities. The other 16 disks remain unused.

And last but not least: Such changes shouldn't be made while the servers are in production. The risk that something goes wrong is too high. If all preconditions are met and there are no problems, it usually doesn't affect the running VMs. But you shouldn't count on that and do it better in a maintenance window.

So I recommend that you read a bit more into vSAN, check in advance that everything is supported, and make the changes in a maintenance window.

Introduction to vSAN

VMware vSAN

[kaschenberg]

I'm certain the hardware is on the HCL:

HP DL380 Gen9 with HP P440 raid controller and all HP enterprise drives.

I've read up on vsan, but it appears my configuration is a bit different than the standard "2 node cluster" that most people are configuring on all these "blog" sites (which BTW is not a real world scenario, like mine is).

Any input is greatly appreciated.

Thank you.

[sk84]

I have edited my first post to be a bit more precise and I have also dealt with the disk groups.

I've read up on vsan, but it appears my configuration is a bit different than the standard "2 node cluster" that most people are configuring on all these "blog" sites (which BTW is not a real world scenario, like mine is).

It has nothing to do with real world scenarios. There are only certain vSAN setups that are recommended or supported and there are technical limitations. 2 Node vSAN setups are primarily intended for ROBO deployments or small infrastructures. And a 2 Node vSAN cluster needs a witness host or appliance.

vSAN and ESXi are interdependent, because vSAN is implemented on hypervisor level. So if you want to make a 2 node vSAN cluster, you have to make a 2 node ESXi cluster. Or you stay with your 5 node cluster, but then all 5 hosts must be configured for vSAN.

[kaschenberg]

I guess I was also hoping for someone to chime in that they've done this setup before and could give me more insight to what I'm trying to accomplish.

I'll wait a bit more for replies.

[TheBobkin]

Hello kaschenberg​,

"2 of these hosts have 24 x 900gb drives and each with 1 x 800gb ssd."

The most storage you will be able to utilise here is 7x900GB per node as Sebastin pointed out already - you could consider getting more SSDs and creating more Disk-Groups with the remaining Capacity-tier disks but if doing this is would be advised to create them on the other hosts to spread the load better (and have more available Fault Domains).

"I figured, why not create a vsan with these unused drives, right?!"

While creating a vSAN cluster is (intentionally) incredibly simple this doesn't necessarily mean one should do so before doing adequate research - I could own a car, not know how to drive, but I can turn on the ignition just fine, this means I should go for a drive right?

"So, I walk through the vsan configuration to enable it, claim disks on both hosts for capacity and caching tier and put host A in 1 fault domain and host B in 1 fault domain to separate them. Now, I'm at the "Finish" step and before I proceed and click the button, does anyone know if this will work?"

No because you don't have a Witness nor a 3rd 'Fault Domain'(FD) for placing components - if you want to use the main benefit of vSAN (redundancy/FTT) then you require 3 FDs, when FDs are not defined the nodes (contributing storage) themselves are considered FDs by default. One only would be setting FDs in a non-stretched cluster if some form of rack splitting was desired (e.g. 3 FDs of 2 nodes each - 2+2+2). Whether one uses a Witness or a node contributing storage, 3 FDs are required for component placement of a FTT=1 RAID-1 Object - sure you *could* make a cluster with only 2 nodes with storage but you could only make FTT=0 Objects so this is not really much use other than for easily recreated/disposable VMs.

This is what a cluster with only 2 nodes providing storage results in using an FTT=1 SP (esx-03a has no storage):

"Also, being that this is a production environment, will anything go "down" on these hosts when the vsan configuration is applied?"

Probably not unless you have something bad like unsupported driver/firmware/disk-access mode set on the controller or are logging to vsanDatastore but this likely wouldn't manifest until you move load to the vsanDatastore. That being said: enabling major changes when you don't understand the possible consequences in Production is universally frowned upon -  this is what Test/Dev is for.

"I'm certain the hardware is on the HCL:

HP DL380 Gen9 with HP P440 raid controller and all HP enterprise drives."

Please check the part/model numbers of all drives on the HCL, vSphere-HCL does not equal vSAN-HCL and we don't universally support drives just because they are 'enterprise-grade', also all SSDs are not created equal, many lower endurance/performance models are not supported for the role of Cache-tier (even for Hybrid) and may only be certified for Capacity-tier role.

If you have trouble navigating the 'Build your own' part lists then try using this tool my colleague made:

https://hcl.captain-vsan.com/

"And one last question, can a 2 node vsan configuration reside in a cluster with 5 hosts in it?"

It wouldn't be a 2-node cluster, configuring a real 2-node cluster fails to even start configuration if there are more than 2 hosts in the cluster. vSAN would be enabled on all 5 nodes in the cluster even if just 2 were providing storage, this also means all 5 hosts would require sufficient vSAN licensing which in my opinion is a bit wasteful/expensive considering how little capacity you would be getting out of it in your current configuration.

"I guess I was also hoping for someone to chime in that they've done this setup before and could give me more insight to what I'm trying to accomplish."

Not to be asinine but making decisions that could impact Production workloads based on strangers on the internet saying 'I did this and it was fine' is not a great strategy - I have seen clusters configured horribly/unsupported and they may work until they don't and then it gets messy - just because you can do something doesn't automatically make it a wise decision.

Bob