ESXi Host NIC Flapping

[rajeevsrikant]

I faced the below after upgrading from NSX version 6.2.2 -> 6.3.2

In each ESXi hosts we have 4 NICs, out of which 2 are used & the remaining 2 are not used.

So we manually disabled the unused NICS.

After the NSX upgrade, in which the new VIBs are installed, the server was put in maintenance mode & got restarted.

After that i observed that the disabled NICs are showing Admin status - Up

So wanted to understand, when the vibs are getting installed during the NSX upgrade, is there any mechanism that it does anything on the NIC settings.

Does it make changes to the available NICs

[Sreec]

After disabling the NIC,did you save the config to ensure it is persist across reboots ?  OR did you disable it from BIOS ? If not NIC will come online after host reboot.

[rajeevsrikant]

I used the below command to down the NIC

# esxcli network nic down -n vmnicX

[Sreec]

That way NIC will be down as long as host is up&running. Like i said either you need to disable it from Host Bios if there is a option or follow http://www.virtuallyghetto.com/2011/06/how-to-disable-vmnic-in-esxi.html  ( add respective entry for NIC  /sbin/vmkchdev -p xxxxxxx & vim-cmd hostsvc/net/refresh in rc file and run /sbin/auto-backup.sh)

[rajeevsrikant]

Thanks.

Does this mean by only with that command, when the server restarts the NIC will come up ?

[Sreec]

Your command is not persistent across reboots . This is expected behavior in ESX/ESXI .However if you change the status at BIOS level ,ESXI wont change the status also the set of the commands which i shared will ensure NIC status is persistent across reboots.

[rajeevsrikant]

Thanks.

So it is nothing to do with NSX upgrade & it is because of the reboot of the server.

[rajeevsrikant]

Also its been mentioned that it is undocumented utility.

What is the VMware official way of doing it ?

[Sreec]

True. You can test this on any host with/without NSX  vibs. I'm not sure about what is the official way. But i always disable unwanted devices from BIOS/Management console which is a safe approach.

[Sreec]

Did you test the steps ? Let me know how it goes.

[rajeevsrikant]

I did the below test

1. Before NSX upgrade I rebooted one of the host servers.

2. After reboot checked the status of the unused NICs, the NICs admin status was down.

3. Upgraded the NSX in that host. As part of upgrade rebooted the host.

4. After reboot when checked the status of the unused NICs the NICs admin status was up.

[Sreec]

Will i was asking you if followed any of the option to disable the NIC and test the host reboot and check the status ? It should work

[rajeevsrikant]

No I was not able to do it what u mentioned.

But what I found is that after NSX Vib installation this is happening.

[Sreec]

It's OK. You can test this on any environment (Nested/Baremetal) platform With/Without NSX result would be same ,because what we disable from hypervisor will never remain persistent across reboots.