Adding in an Uplink Port on a vDS causes Fails due to Host Offline Potential.

[dlaemmle]

I have an ESX 5.1 cluster of 3 ESX 5.1 Hosts,

The vDS (5.1) is set up so that it has 1 uplink port only that is tied to the vmnic0 on the hosts.

I want to add in a second uplink port tied to vmnic2 that will be used to RSPAN traffic out to a remote traffic analyzer.

I have all NIC teaming, and failover turned off for the switch and port group.

I Create a new uplink port on the vDS without a problem,

Then I associate vmnic2 to the the  new Uplink port. When I press save, the task goes off and tries to propagate this change to the host.

But I get a failure warning me that this action would take the Host off line, so it rolls back the change

I can associate the Uplink Port to the Physical NIC if I log onto the ESX CLI and set the NIC down.

$esccli network nic down -n vmnic2

Then I can use the GUI to associate that down vmnic2 to the port group.

However if I then set that NIC to active, the ESX Host disconnects from the Cluster and I have to use the Console to reset the VDS and and force the NIC to use the proper Uplink port.

However Both adapters appear to share the same IP. The RSPAN uplink port is connected in such away that no traffic or ipaddress should get set on this NIC, also in some cases I have tried to Uplink NIC is directly connected to a traffic analyzer.

This appears to work Although I have some odd routing and DHCP issues for some VMs (Some work great) and hosts -- I'll write more on this in a different post. 

This whole setup seems incorrect. I can't find any documentation that states how to add an uplink port to an already existing vDS.  I do find documentation that states "uplink ports can be added later ".

How do I add an Uplink port and associate it to a Host's interface for RSPAN ?  This is one of the big new additions to 5.1 and it seems very difficult/impossible to set up correctly.

[dlaemmle · Accepted solution]

I'll answer my own question.

In my steps above, before I "Then I associate vmnic2 to the the  new Uplink port. When I press save, the task goes off and tries to propagate this change to the host" , I need to break the NIC failover configuration that I never set up and thought that I had specifically disabled for my vds. But that is not how it is done.

Before we can associate a physical NIC on the host to the new Uplink port, we need to make sure the uplink port is not being network teamed. This is a policy that is set at the Port Group. So each Port group in the vDS must be modified. There likely a way to set this as a standard Port group policy, but for now we’ll do it per port group.

Despite turning off Nic Teaming, and LACP (must be turned off for port mirroring), vmWare automatically tries to load balance VMs across active Uplink ports. This can cause problems other than for mirroring  (I had both VM Nics on an EH1000v get associated to the same uplink port at one point) .

In the Edit Settings panel, select the Teaming And Failover menu item. The new Mirroring Uplink port will be listed in the “Active uplinks” with the other uplink ports. Click on the newly created uplink port and use the Arrows to move the port to the “Unused uplinks”. This does not mean that port is not used, just not used in the failover/load balancing rules described in the top of the dialog.

Now you can go Add a physical nic on Each Host to the VDS and associate to the Span out port