vmwarelicense12
Enthusiast
Enthusiast

VSAN on two distributed switched

Jump to solution

We have 8 ESXi hosts in one vSAN cluster using one dvs.

Is it possible to configure 4 of the hosts to use one dvs for vsan and the 4 other hosts to use another dvs (same vsan vlan)?

0 Kudos
1 Solution

Accepted Solutions
AlexanderLiucka
Enthusiast
Enthusiast

vmwarelicense123,

Yes, you can.

View solution in original post

0 Kudos
8 Replies
AlexanderLiucka
Enthusiast
Enthusiast

vmwarelicense123,

Yes, you can.

View solution in original post

0 Kudos
vmwarelicense12
Enthusiast
Enthusiast

This is our setup.

3 esxi in one cluster, where each have:

2 uplinks on dvs1 used for management, vmotion, vsan, vxlan (all)

2 uplinks on dvs2 witch I whant to use for vsan.

I then try to migrate vmk2 (vsan) from dvs1 to dvs2. The kernel port is migrated, but reports that the host cannot se the other vsan nodes.

What am I doing wrong?

0 Kudos
AlexanderLiucka
Enthusiast
Enthusiast

Check the health status of your dvs2. I'm almost sure that you have not configure correctly the vsan vlan on the hardware switch to which go the 2 uplinks of dvs2.

0 Kudos
vmwarelicense12
Enthusiast
Enthusiast

ohh. dear.. ohh... dear.. I forgot to set MTU to 1600 on the new dvs!!

MTU is 1600 om the old dvs i migrate from, so now it all work at expected.

0 Kudos
AlexanderLiucka
Enthusiast
Enthusiast

This is strange.

When you have wrong MTU, the vsan health will reportit as incorrect MTU size which is very different from "the host cannot see the other vsan nodes".

Also it is very strange that, you have set the MTU at 1600 not at 9000? the Default MTU is 1500 which is very close to your 1600.

0 Kudos
TheBobkin
VMware Employee
VMware Employee

Hello Alexander,

They are likely using NSX which requires 1600 MTU.

Bob

0 Kudos
vmwarelicense12
Enthusiast
Enthusiast

Hi TheBobkin,

Exactly... we are using NSX;)

0 Kudos
AlexanderLiucka
Enthusiast
Enthusiast

Hello TheBobkin,

You are correct that they use vxlan and for vxlan the minimal MTU is 1600 but they can't put vSan, vMotion, iSCSI, NFS, FCoE, Management, etc over vxlan.

This mean that the vsan and the vxlan are on the same level. To work vxlan the dvs has to have minimum MTU of 1600 or higher, but for the vsan the default MTU of 1500 is enough.

Again I don't know how vmwarelicense123 will receive the error that the vsan host can not see the other vsan nodes when you can not put vSan, vMotion, iSCSI, NFS, FCoE, Management, etc over vxlan which was not working for the wrong MTU?

P.S.

https://storagehub.vmware.com/t/vmware-vsan/vmware-r-vsan-tm-network-design/nsx/

VMware Virtual SAN and VMware NSX Compatibility - Virtual Blocks

0 Kudos