VMware Cloud Community
mshipek
Contributor
Contributor
Jump to solution

VM VLAN networking

Greetings,

I had another post that was rather windy and I apologize.  I have since done more research and gotten farther along.

I am having difficulty understanding how the gateways on the vmkernels work, specifically with regard to the management network, vmotion and iscsi segmented networks.

I have my iSCSI on the 10.101.12.x network, VLAN 12, my vmotion on the 10.101.10.x network, VLAN 10 and my management on 10.101.100.x, VLAN 100.  My VMs are on 10.101.11.x.  Whenever I change the gateway for any of these connections so that they can communicate outside their subnet per standard network, it changes the gateway on ALL of the vkernal adapters.  Which gateway does VMware like to use and can anyone explain how it can route outside of the subnet it's on if it doesn't have a next hop?  I understand VLAN adjacency for all the VLANs, but the default gateway is necessary to get outside the VM networks and VLANs.

Thank you in advance,

Max

Reply
0 Kudos
1 Solution

Accepted Solutions
rcporto
Leadership
Leadership
Jump to solution

So if I understand you correctly, are you saying that the vmotion VMkernel and iSCSI VMkernel must be on the same subnet?  because right now, I have the vmotion on a different subnet than everything else.  It is on VLAN 10 (10.101.10.x) and the iSCSI is on VLAN 12 (10.101.12.x).  My VMs are on a different subnet as well, VLAN 11 (10.101.11.x).

Sorry, but you misunderstand or I not explain so good 😉 All vMotion VMkernel must be on a single flat network and the iSCSI VMkernel on another (or the same) single flat network... but don't place one vMotion VMkernel ina host in one subnet and on another host on a different subnet... and in iSCSI network, the initiator (the ESXi host VMkernel adapter) must be on the same network of target (the storage).


When you say "Since you don't need default gateway on vMotion and iSCSI network, because they are on the same subnet" I understand this as they all need to be on the same physical network i.e. 10.1.1.x without any VLAN separation.  Is that correct?

No, they can be on different network and different VLAN, cause these network don't need routing... and you only need a default gateway if you need send packets to a different network.

For example, in a environment with two ESXi host, you can have the following:

ESXi01:

iSCSI - 10.101.12.1 - VLAN 12

vMotion - 10.101.10.1 - VLAN 10

Management - 10.101.100.1 - VLAN 100

VM Network - 10.101.11.x

ESXi02:

iSCSI - 10.101.12.2 - VLAN 12

vMotion - 10.101.10.2 - VLAN 10

Management - 10.101.100.2 - VLAN 100

VM Network - 10.101.11.x

And the default gateway need to be configured only on Management VMkernel adapter.

---

Richardson Porto
Senior Infrastructure Specialist
LinkedIn: http://linkedin.com/in/richardsonporto

View solution in original post

Reply
0 Kudos
4 Replies
rcporto
Leadership
Leadership
Jump to solution

The vMotion VMkernel network must be on the same subnet on source and destination hosts... and iSCSI VMkernel adapters must be on the same subnet as the iSCSI target.

Since you will don't need default gateway on vMotion and iSCSI network, because they are on the same subnet, you need to setup default gateway only on Management VMkernel adapter.

Here are some guidelines/best practices for vMotion and iSCSI:

VMware KB:    Creating a VMkernel port and enabling vMotion on an ESXi/ESX host

VMware KB:    vMotion migrations fail when using multiple VMkernel ports for vMotion in different IP...

https://pubs.vmware.com/vsphere-55/index.jsp?topic=%2Fcom.vmware.vsphere.storage.doc%2FGUID-66AF5FA0...

---

Richardson Porto
Senior Infrastructure Specialist
LinkedIn: http://linkedin.com/in/richardsonporto
Reply
0 Kudos
mshipek
Contributor
Contributor
Jump to solution

Thank you for your response Richardson.

So if I understand you correctly, are you saying that the vmotion VMkernel and iSCSI VMkernel must be on the same subnet?  because right now, I have the vmotion on a different subnet than everything else.  It is on VLAN 10 (10.101.10.x) and the iSCSI is on VLAN 12 (10.101.12.x).  My VMs are on a different subnet as well, VLAN 11 (10.101.11.x).  I initially had everything on a single flat network, but after learning more and researching best practices, I've understood separating out these roles into VLANs is recommended.

When you say "Since you don't need default gateway on vMotion and iSCSI network, because they are on the same subnet" I understand this as they all need to be on the same physical network i.e. 10.1.1.x without any VLAN separation.  Is that correct?

Reply
0 Kudos
rcporto
Leadership
Leadership
Jump to solution

So if I understand you correctly, are you saying that the vmotion VMkernel and iSCSI VMkernel must be on the same subnet?  because right now, I have the vmotion on a different subnet than everything else.  It is on VLAN 10 (10.101.10.x) and the iSCSI is on VLAN 12 (10.101.12.x).  My VMs are on a different subnet as well, VLAN 11 (10.101.11.x).

Sorry, but you misunderstand or I not explain so good 😉 All vMotion VMkernel must be on a single flat network and the iSCSI VMkernel on another (or the same) single flat network... but don't place one vMotion VMkernel ina host in one subnet and on another host on a different subnet... and in iSCSI network, the initiator (the ESXi host VMkernel adapter) must be on the same network of target (the storage).


When you say "Since you don't need default gateway on vMotion and iSCSI network, because they are on the same subnet" I understand this as they all need to be on the same physical network i.e. 10.1.1.x without any VLAN separation.  Is that correct?

No, they can be on different network and different VLAN, cause these network don't need routing... and you only need a default gateway if you need send packets to a different network.

For example, in a environment with two ESXi host, you can have the following:

ESXi01:

iSCSI - 10.101.12.1 - VLAN 12

vMotion - 10.101.10.1 - VLAN 10

Management - 10.101.100.1 - VLAN 100

VM Network - 10.101.11.x

ESXi02:

iSCSI - 10.101.12.2 - VLAN 12

vMotion - 10.101.10.2 - VLAN 10

Management - 10.101.100.2 - VLAN 100

VM Network - 10.101.11.x

And the default gateway need to be configured only on Management VMkernel adapter.

---

Richardson Porto
Senior Infrastructure Specialist
LinkedIn: http://linkedin.com/in/richardsonporto
Reply
0 Kudos
mshipek
Contributor
Contributor
Jump to solution

Awesome, thank you!  That's what I have it configured now as and it seems to be working.

Reply
0 Kudos