Change vswitch to use a different vmnic?

[tdhaslett]

Hi All,

I am starting to pull out what hair I have left over here! I have been trying to figure this out for a few days now and have tried the steps below on two of my hosts unsuccessfully..

I have 5 ESXi 6.7 hosts. On each of them, the Management network (vmk0) is on vswitch0, which is uplinked using vmnic0. I need to re-use vmnic0 for adding a new network to the hosts. I have a 10GB port (vmnic2) unused that I want to move vmk0 & vswitch0 over to, since my new network does not have any 10GB ports available.

I have connected vmnic2 and set the VLAN on the physical switch port accordingly, and changed vmnic2 to be Uplink 1. I have also followed the instructions in this document to set the Management vmKernel to use the MAC of vmnic2, but once I disconnect the patch going in to vmnic0, I cannot communicate with the host until I plug it back in. It seems that for some reason, I cannot change the host over to using vmnic2 for vswitch0.

What am I missing that I am not doing correctly?

Thanks!

Tim

[daphnissov]

You might not be missing anything. You may have a physical networking (L2 or L3) issue that exists beyond the scope of ESXi. What you may want to do is use that vmnic2 (10GbE) to create a new vswitch on the host with a new vmkernel port. It doesn't have to be used for anything, but it'll give you a virtual interface. If you can't ping that vmkernel port exterior to your ESXi host you have a physical networking problem you must fix.

[tdhaslett]

Ah, good idea! I'll give it a go and see what happens.

Thanks!

[tdhaslett]

Well, now I'm baffled. I've set up a new vswitch and vmkernel and connected it to vmnic2. ESXi sees the name of the switch and the switch port that it is connected to, and the switch can see the MAC of vmnic2 (using show mac address-table interface...). However, the switch's ARP table shows incomplete instead of the MAC of vmnic2.

I have tried this on two hosts, connected to two different switches, with multiple SFP+ cables, including fiber with GBICs and twinax, and set the ports on the switches to multiple different VLANs.

I think my issue is in the config of the nic, although I have no idea what needs to be changed. It's a Cisco UCS VIC 1225 and I am connecting to Cisco switches - one is a 3750X-48P with an SFP module, the other is a 3850 12XS switch.