2 Node Direct Connect Questions

[ArchFan08]

We already have a 3 node VSAN 6.2 Hybrid cluster at what we'll call 'Site A' that's been running well for about 6 months. I just assembled a 2-node VSAN 6.5 Hybrid cluster at 'Site B' that's leveraging the newly added direct-connect support using dual 10Gb NICs for vMotion and VSAN traffic between the 2 hosts. Each site has its own vCenter Server appliance. Sites are connected via site-to-site VPN tunnels.

I understand that I need to deploy a witness appliance and my plan is to do that on the VSAN cluster at 'Site A' - how do I go about doing this? I understand how to deploy the OVA, but how does the cluster at 'Site B' know about the witness - do I configure that in the Stretched Cluster portion of VSAN configuration on 'Site B'?

[TheBobkin · Accepted solution]

Hi there!

Note that you must have below 500ms RTT Network latency between the data site and the Witness.

The Witness appliance is basically a VM that is then registered in vCenter inventory as an appliance (appears like a host icon but blue).

Decent GUI-based guide for setting up Witness:

http://cormachogan.com/2015/09/14/step-by-step-deployment-of-the-vsan-witness-appliance/

After setting up the Witness:

In the Web Client go to vSAN Cluster > Manage > Fault Domains and Stretched Cluster > Configure, then go through the Wizard to configure FDs and select the Witness.

Or

CLI-based guide:

https://blog.rylander.io/2017/01/19/configure-2-node-vsan-on-esxi-free-using-cli-without-vcenter

Once deployed you can add it to the cluster as a Witness with CLI via SSH using:

(from one of the existing nodes):

#esxcli vsan cluster get

Note the 'Sub-cluster UUID'

#esxcli vsan faultdomain get

Note the Fault Domain ID

Join the node to the vSAN cluster as a Witness in a specified Fault Domain:

#esxcli vsan cluster join -u <Sub-cluster UUID> -w -t -p <Fault Domain ID>

Link to vSAN esxcli: VMware vSphere 6.5 Documentation Library

The only difference I have noticed between 6.2 and 6.5 version is that you set up Witness traffic separately:

vSAN 2 Node with Direct Connect - Virtual Blocks

Bob

-o- If you found this comment useful or answer please select as 'Answer' and/or click the 'Helpful' button -o-

[ArchFan08]

I think I understand now, only one remaining question. When configuring the stretched cluster options at Site B, it asks for the location of the witness appliance and allows me to select from the hosts listed under Site B's vCenter. Does that mean that I register the witness appliance located at Site A in Site B's vCenter rather than Site A's?

[TheBobkin]

Typically (if you are going multi-site as you suggested here) you register the Witness Appliance at site A as a VM (this is where it is really running), then you register this as an appliance in vCenter on site B, at the datacenter level not in the vSAN cluster (either the same DC as the vSAN cluster or its own DC)

More info (not too much other than Witness pg has changed since 6.2:

http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/products/vsan/vmware-virtual-san-6....

Bob

-o- If you found this or any other comment useful or answer please select as 'Answer' and/or click the 'Helpful' button , please ask follow-up questions if you have any -o-

[ArchFan08]

Thanks. I think the nested ESXi host was throwing me off and I was overcomplicating it - I've got the witness 'host' added to vCenter at Site B now. I need to separate the witness traffic, and looking here: vSAN 2 Node with Direct Connect - Virtual Blocks  I understand that I need to tag a vmkernel port on each host at Site B for witness traffic (I'm thinking of just tagging the existing management vmkernel since they'd be on the same VLAN anyway and the route already exists unless there's a reason I shouldn't), but what about the witness appliance - do I need to tag one of them for witness traffic, or just assign the existing VSAN Network vmkernel an IP?

[ArchFan08]

Jase McCarty clarified this for me by pointing me here: Storage and Availability Technical Documents

[Jasemccarty]

Glad I could help ArchFan08