Good Afternoon!
I have recently purchased a VMUG license as I work in the IT field and have dealt with VMware a lot in my previous positions but my current job doesn't have VMware and I want to get my fix so I am building a home lab. Because it's a home lab I obviously will not have more than a single site but I would like to set up vSAN to prevent having to buy additional hardware and I have never had the chance to use vSAN. I am trying to figure out what would be the best in my situation a 2 or 3-node setup.
Ideally, I would like to do a 3 Node setup and place a single 2-port 40GB or 100GB card in each node and have each node connected to each other and have all vSAN and vMotion traffic go over there. In theory, as long as they all shared the same IP space and they all connected to each other it should work without issue. Are there any limitations to this? Doing a 3-node system would prevent me from having to have a separate physical witness correct?
If that will not work, can I do the same networking connection (Direct connection) in a 2-node setup if I have a physical witness able to talk to them via the normal connection ports?
@HellBomb, daisy-chaining 3 nodes without a switch won't reliably work and isn't supported, some better details of this in the comments here: https://www.reddit.com/r/vmware/comments/fljb6p/can_one_create_a_3_node_vsan_without_a_switch/
So yes, go with 2-node direct connect and then either a 3rd physical node configured as Witness with WTS configured over Management network or you could even configure the 3rd server as normal ESXi running a Witness Appliance (and any other workloads). Or if you have a switch then go with 3-node using this.
Awe interesting, so could I do a 2 vSAN + 1 compute node running? The two vSAN servers can be connected directly to each other, and then all 3 servers can be connected via 1GBE ports to the network switch.
@HellBomb Yes, the 2 data-nodes can be directly connected without a switch and then both of these connected to the Witness Appliance (or physical node configured as Witness if you wanted to but I don't see the benefit of this) via 1Gb switch.
https://core.vmware.com/resource/vsan-2-node-cluster-guide#sec7423-sub1
Just for clarification, you said "2 vSAN + 1 compute" but the Witness Appliance is just a VM running on a server, it doesn't support running VMs on it so it doesn't provide 'compute' to the cluster, though yes the physical server hosting the Witness Appliance can of course run other VMs and provides 'compute' resources, that server is not a node in the cluster.
