Problems with deploying 2-node vSAN cluster at remote office

[ValterFE]

Hello,

I cannot figure out what am I doing wrong, so now I am starting this remote site setup from scratch and I am doing it step-by-step. First my topology:

Site A:

4 node in vSAN cluster

VLAN01 MGMT (10.0.1.0/24) - vmk0

VLAN02 vSAN (10.0.2.0/24) - vmk1

VLAN03 vMOTION (10.0.3.0/24) - vmk2

vSAN on site A works w/o any problems.

Site B:

2 node

VLAN01 MGMT (10.1.1.0/24) - vmk0

VLAN02 vSAN (10.1.2.0/24) - vmk1

VLAN03 vMOTION (10.1.3.0/24) - vmk2

Hosts on site B are configured to carry witness traffic over management vmk0 (esxcli vsan network ip add -i vmk0 -T=witness)

Witness host placed on Site A:

vmk0 Management Network on vSwitch0 connected to VLAN01 (IP: 10.0.1.10)

vmk1 witnessPg on witnessSwitch - where to connect this vmk?

In configuration assistant I get following errors:

- Network configuration

1. vSAN cluster partition

2. vSAN: Basic (unicast) connectivity check (all hosts on site B and witness host on both vmk0 and vmk1)

3. vSAN: MTU check (ping with large packet size (all hosts on site B and witness host on both vmk0 and vmk1)

I can ping/vmkping:

Witness host from both servers on site B (only management IP address)

ping 10.0.1.10

PING 10.0.1.10 (10.0.1.10): 56 data bytes

64 bytes from 10.0.1.10: icmp_seq=0 ttl=62 time=0.413 ms

vmkping -s 9000 10.0.1.10

PING 10.0.1.10 (10.0.1.10): 9000 data bytes

9008 bytes from 10.0.1.10: icmp_seq=0 ttl=62 time=0.830 ms

Servers on site B from witness host (only mnagement IP address)

ping 10.1.1.11

PING 10.1.1.11 (10.1.1.11): 56 data bytes

64 bytes from 10.1.1.11: icmp_seq=0 ttl=62 time=0.470 ms

vmkping -s 9000 10.1.1.11

PING 10.1.1.11 (10.1.1.11): 9000 data bytes

9008 bytes from 10.1.1.11: icmp_seq=0 ttl=62 time=0.769 ms

Questions:

1. Where to connect witnessPg vmk?

2. Why do I need witnessPg vmk if witness traffic runs over management network?

3. Do I need to add any static routes and if yes from where to where?

4. Under disk management, witness host disk are in Group 1, while disks of the 2 hosts are in Group 2, should this be like this?

[ValterFE · Accepted solution]

Problem resolved!

I could not get management vmk to carry witness traffic, so....

1. Use direct connect between 2 nodes on Site B for vSAN

2. Additional L3 network and VLAN04 to carry ONLY witness traffic

2. Add additional vmk to 2 nodes in Site B (10.1.4.0/24)

3. Tag vmk3 (step 2) to carry witness traffic

4. witnessPg vmk on witness host connected to the VLAN04 (10.0.4.0/24)

5. Static route from witness host to hosts on Site B using VLAN04 gateway on Site A

5. Static route from hosts on Site B to witness host using VLAN04 gateway on Site B

6. Enable vSAN

Hope this helps someone..