VMware Cloud Community
MJMSRI
Enthusiast
Enthusiast
‎10-03-2018 01:29 AM

Help with vSAN 2-Node Direct Connect vSAN 6.7 Networking

Hi All,

I have looked though storage hub and the 2node/stretched cluster documents but still cannot get a firm answer to the networking on a new 2-Node Cluster.

The scenario is as below:

  • Everything will be hosted in one location on one site so all Layer 2 connected across same Cisco Switches. There is an existing vSphere 5.5 cluster and there will be a new vSAN 6.7 2-Node Direct Connect Cluster. They will all use the same VLAN100 for VMware Management. So all hosts in business and vCenters, etc will have same 10.10.0.0/16 subnet IP’s.
  • Existing vSphere cluster is ESXi 5.5 and this is where the new vSAN Virtual Appliance will be hosted as a VM.
  • New cluster will be vSAN 6.7 2-node direct connect over 2 interfaces that are 10gbE. So this will have vSAN VMKernel on it and vMotion VMKernel. Setup in Active/Standby Configuration (vSAN = vmnic4 active, vmnic5 standby. vMotion = vmnic5 active, vmnic4 standby)
    • As this is a switchless solution for the vSAN and vMotion networks im not sure if I need to specify VLAN’s on the vSAN and vMotion Networking? Or as they will be on 2 different VMKernel Adapters that will suffice?
    • I have made up two VLAN’s that these networks will use so these dont exist anywhere yet such as Cisco Switches, etc.
  • The onboard 4 x1gbE interfaces in new vSAN Hosts will be used as 2 for VMware Management and 2 for Virtual Machine Networks.
  • As this will be vSAN 6.7 and all Layer 2 in the same site I don’t believe we will need to specify any static routes for the traffic between the 2-Node and the witness on other cluster?

  • Part I want to clarify is the networking from the vSAN Cluster to the vSAN Witness. So I see that the Witness is deployed with 2 vNICS, vmnic0 for MGMT and vmnic1 for WitnessPG however its supported to enable vSAN Networking across the vmnic0 MGMT so think I will do that in which case this means that the vSAN Networking will communicate over the MGMT Network?
  • If that’s the case then the VLAN500 I have made up that the vSAN will communicate only over the direct-connections and does not need to be routed any where else such as to the vSAN Appliance?

Or is there more to this?

0 Kudos
1 Reply
GreatWhiteTec
VMware Employee
VMware Employee
‎10-05-2018 05:38 AM

Hi MJMSRI,

I believe you are missing a piece. The witness traffic will try to go out the vSAN nodes on the vSAN vmk, unless you specify otherwise. Since those are direct connect, they won't know how to get out. You can use Witness Traffic Separation to send witness traffic (not much) out the management interface of the vSAN nodes. See WTS in StorageHub

Witness Traffic Separation (WTS)

From the witness side you can use the vmk tagged for witness traffic or use tag the mgmt vmk for that. You shouldn't have to add static routes if they are all on the same segment.

For your vSAN/vMotion traffic, I'll recommend enabling NIOC via vDS. The vDS switches are included with all vSAN licenses regardless of your vSphere licenses. In case you lose a NIC, all traffic will flow through one NIC, and vMotion may cause issues if there is contention. See StorageHub for NIOC share configuration.

Enabling NIOC

Hope this helps.

0 Kudos