Noein
Enthusiast
Enthusiast

Disabling vMotion for vmnic

Hello! I have a problem with vMotion. I have 2 servers, 2 vmnics on each server. On vmnic0 i have "Manadgement" and "VM Network". On vmnic1 i whant to configure vMotion. But no matter what i do, when i start moving VM from one host to another, all traffic goes through vmnic0.

VCSA: 6.7.0.10000

Configuration of both servers when vmk1 use vMotion TCP stack:

 

Spoiler

Server 1: VMware ESXi, 6.7.0, 8169922
esxcfg-nics -l
Name PCI Driver Link Speed Duplex MAC Address MTU Description
vmnic0 0000:04:00.0 ixgben Up 1000Mbps Full e4:11:5b:96:56:b8 1500 Intel Corporation 82599EB 10 Gigabit Dual Port Backplane Connection
vmnic1 0000:04:00.1 ixgben Up 1000Mbps Full e4:11:5b:96:56:b9 9000 Intel Corporation 82599EB 10 Gigabit Dual Port Backplane Connection

esxcfg-vmknic -l
Interface Port Group/DVPort/Opaque Network IP Family IP Address Netmask Broadcast MAC Address MTU TSO MSS Enabled Type NetStack
vmk0 Management Network IPv4 192.168.1.225 255.255.255.0 192.168.1.255 e4:11:5b:96:56:b8 1500 65535 true STATIC defaultTcpipStack
vmk1 vMotion Network IPv4 192.168.42.225 255.255.255.0 192.168.42.255 00:50:56:61:95:08 9000 65535 true STATIC vmotion

Server 2: VMware ESXi, 6.7.0, 8169922
esxcfg-nics -l
Name PCI Driver Link Speed Duplex MAC Address MTU Description
vmnic0 0000:04:00.0 ixgben Up 1000Mbps Full e4:11:5b:95:79:84 1500 Intel Corporation 82599EB 10 Gigabit Dual Port Backplane Connection
vmnic1 0000:04:00.1 ixgben Up 1000Mbps Full e4:11:5b:95:79:85 9000 Intel Corporation 82599EB 10 Gigabit Dual Port Backplane Connection

esxcfg-vmknic -l
Interface Port Group/DVPort/Opaque Network IP Family IP Address Netmask Broadcast MAC Address MTU TSO MSS Enabled Type NetStack
vmk0 Management Network IPv4 192.168.1.217 255.255.255.0 192.168.1.255 e4:11:5b:95:79:84 1500 65535 true DHCP defaultTcpipStack
vmk0 Management Network IPv6 fe80::e611:5bff:fe95:7984 64 e4:11:5b:95:79:84 1500 65535 true STATIC, PREFERRED defaultTcpipStack
vmk1 vMotion Network IPv4 192.168.42.217 255.255.255.0 192.168.42.255 00:50:56:64:0c:b8 9000 65535 true STATIC vmotion
vmk1 vMotion Network IPv6 fe80::250:56ff:fe64:cb8 64 00:50:56:64:0c:b8 9000 65535 true STATIC, PREFERRED vmotion

 


When I configure vmkernel adapters with "Default TCP Stack" with vMotion enabling vmkping from vmk1 is working in both directions:

Spoiler

vmkping -I vmk1 192.168.42.217 -d -s 8972
PING 192.168.42.217 (192.168.42.217): 8972 data bytes
8980 bytes from 192.168.42.217: icmp_seq=0 ttl=64 time=1.004 ms
8980 bytes from 192.168.42.217: icmp_seq=1 ttl=64 time=0.964 ms
8980 bytes from 192.168.42.217: icmp_seq=2 ttl=64 time=0.979 ms

--- 192.168.42.217 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.964/0.982/1.004 ms

vmkping -I vmk1 192.168.42.225 -d -s 8972
PING 192.168.42.225 (192.168.42.225): 8972 data bytes
8980 bytes from 192.168.42.225: icmp_seq=0 ttl=64 time=1.072 ms
8980 bytes from 192.168.42.225: icmp_seq=1 ttl=64 time=1.002 ms
8980 bytes from 192.168.42.225: icmp_seq=2 ttl=64 time=1.030 ms

--- 192.168.42.225 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 1.002/1.035/1.072 ms

 

In all cases vMotion on vmk0 is disabled.

MTU on vSwitches, VMKernel adapters and on Catalyst Blade Switch 3120 is 9000. Please, tell me what am i doing wrong 😞

Why vMotion dont whant to flow through vmk1 and 192.168.42.0/24 network? 😞

0 Kudos
7 Replies
depping
Leadership
Leadership

can you show the output of, assuming you use a regular vSwitch with that name:

esxcli network vswitch standard policy failover -v vSwitch0

depping
Leadership
Leadership

As I am guessing both NICs are attached to the same vSwitch and it is "active/active" load balancing

Noein
Enthusiast
Enthusiast

On both servers output is same, one vmnic on one switch 😞 :

Spoiler
[root@esxi1:~] esxcli network vswitch standard policy failover get -v vSwitch0
Load Balancing: srcport
Network Failure Detection: link
Notify Switches: true
Failback: true
Active Adapters: vmnic0
Standby Adapters:
Unused Adapters:
[root@esxi1:~] esxcli network vswitch standard policy failover get -v vSwitch1
Load Balancing: srcport
Network Failure Detection: link
Notify Switches: true
Failback: true
Active Adapters: vmnic1
Standby Adapters:
Unused Adapters:
0 Kudos
depping
Leadership
Leadership

and your vmotion vmkernel interface is on vSwitch1?

depping
Leadership
Leadership

maybe you can show the vcenter ui, a bit easier for us πŸ™‚

Noein
Enthusiast
Enthusiast

Sure, i can make any screenshot of vcenter ui. Here is screenshot of Host > Configure > Networking > VMkernel adapters from host 192.168.1.225 (vSwitch0 / vmnic0) . Configuration of another host is same, except its ip address is 192.168.1.217 (vSwitch0 / vmnic0).

vMotion is configuring on vSwitch1 / vmnic1 on each server, ip addresses is 192.168.42.217 and 192.168.42.225

vmk1 - ipv4 settings.PNG

vmk1 - settings.PNG

  

vmotion.PNG

 

0 Kudos
Noein
Enthusiast
Enthusiast

I found new very important information, that i thought useless until this night. I found that newly created virtual machine, migrate through vmk1, useing interfaces that is directly connected by cable. But if i try to migrate old VMs, they migrate through vmk0 and dont whant to migrate through vmk1. So i think this new information is important:

I have vSphere Essentials Plus cluster with 2 hosts. Everything works ok for 3 years, but few weeks ago vCenter was unable to recover from power loss with postgresql fatal error. I spent few days with no result with recovering vcenter, so i powered off old vcsa and installed new. Added hosts there, everything work, everything cool.

So now i have virtual machines that was created in old VCSA, that is dead. And this VMs dont whant to use vmk1 for migration, and use vmk0 for everything. VMs that i created in new vcsa have no problems with migrating through vmk1, and everything is cool with them.

Now i will try to unregister and register old vms to test migration agein

Here is results of migrations. green - old VM, red - new VM. Cloned old vm migrate through vmk0 :(. Trying to export it to OVF template and import agein

vmotion graph.PNG

0 Kudos