Its been a while since I last wrote a blog post so I thought I should put up a quick one.

In NSX deployments and designs I have worked on, common questions I get when using a teaming method on NSX-v of Loadbalance-SRCID/MAC is:

 

1. How can I determine what physical interface VXLAN encapsulated traffic will egress/ingress via on an ESXi host?

2. What will be the SRC IP address and SRC MAC address of the VXLAN traffic?

Please note this does not include traffic traversing a DLR, I will cover that in a later post.

So to determine this you can follow the below process:

 

If you run the command esxtop from an ESXi hosts cli, then type n (for network) you will get a list of VM's DVPorts and VMKernel interfaces (VTEPs) and more importantly what DVUplink/vmnic on the host they are mapped to as below shows. In this example I'm going to be using VM04 as an example.

 

So in the below example from esxtop VM04 is using vmnic1 (DVUplink1) on my DVSwitch.

 

Output of esxtop for VM's:

 

  67108876      39870:VM04.eth0     vmnic1 DvsPortset-1         0.00    0.00       1.94    0.00   0.00   0.00

  67108877      49943:wah1.eth3     vmnic1 DvsPortset-1          0.00    0.00       0.00    0.00   0.00   0.00

  67108878      49943:wah1.eth2     vmnic2 DvsPortset-1          0.00    0.00       0.00    0.00   0.00   0.00

  67108879      49943:wah1.eth1     vmnic2 DvsPortset-1          0.00    0.00       0.00    0.00   0.00   0.00

  67108880      49943:wah1.eth0     vmnic1 DvsPortset-1          0.00    0.00       0.00    0.00   0.00   0.00

  67108881      50475:wah2.eth3     vmnic1 DvsPortset-1          0.00    0.00       0.00    0.00   0.00   0.00

  67108882      50475:wah2.eth2     vmnic2 DvsPortset-1          0.00    0.00       0.00    0.00   0.00   0.00

  67108883      50475:wah2.eth1     vmnic2 DvsPortset-1          0.00    0.00       0.00    0.00   0.00   0.00

  67108884      50475:wah2.eth0     vmnic1 DvsPortset-1          0.00    0.00       0.00    0.00   0.00   0.00

  67108885      51009:wah3.eth0     vmnic1 DvsPortset-1          0.00    0.00       0.00    0.00   0.00   0.00

 

If you then run the below esxcli command to list what VTEP a DVPort is using, we can see that VM04 on Switch port 67108876 is using a VTEP with an ID of 0.

 

/var/log # esxcli network vswitch dvs vmware vxlan network port list --vds-name=DSwitch-Cluster2 --vxlan-id=5000

Switch Port ID  VDS Port ID  VMKNIC ID

--------------  -----------  ---------

      67108870  vdrPort             1

      67108876  142                  0    <<< VM04

      67108877  141                  0

      67108878  143                  1

      67108879  144                  1

      67108880  145                  0

      67108883  139                  1

      67108884  140                  0

      67108885  138                  0

 

If we then use the below esxcli command to check what vmknic VTEP ID 0 maps to, this will provide the IP address and MAC address we will SRC traffic from when VM04 traffic is encapsulated in VXLAN from this host.

 

~ # esxcli network vswitch dvs vmware vxlan vmknic list --vds-name=DSwitch-Cluster2

Vmknic Name  Switch Port ID  VDS Port ID  Endpoint ID  VLAN ID  IP          Netmask        IP Acquire Timeout  Multicast Group Count  Segment ID

-----------  --------------  -----------  -----------  -------  ----------  -------------  ------------------  ---------------------  ----------

vmk1               67108871  130                    0        0  172.16.1.5  255.255.255.0                   0                      0  172.16.1.0

vmk2               67108872  137                    1        0  172.16.1.6  255.255.255.0                   0                      0  172.16.1.0

 

~ # esxcfg-vmknic -l <trunkated output>

Interface  Port Group/DVPort   IP Family IP Address        Netmask         Broadcast       MAC Address       MTU     TSO MSS   Enabled Type            

vmk1       130                 IPv4      172.16.1.5                   255.255.255.0   172.16.1.255    00:50:56:68:93:ed 1600    65535     true    STATIC           

 

We can then check again in esxtop what vmnic the VTEP vmk1 also mapped to and see if they matches what VM04 was using (vmnic1)

 

Output of esxtop for vmknic's:

 

~ #

  67108871                 vmk1     vmnic1 DvsPortset-1          0.19    0.00       0.77    0.00   0.00   0.00

  67108872                 vmk2     vmnic2 DvsPortset-1          0.00    0.00       0.97    0.00   0.00   0.00

 

So from the above VM04 is using VTEP ID 0 which is vmk1. Both vmk1 and VM04 are using vmnic1

 

So in conclusion when VM04 sends traffic to be encapsulated in VXLAN:

 

1. Outer SRC IP address is 172.16.1.5 (vmknic1)

2. Outer SRC MAC address is 00:50:56:68:93:ed (vmknic1)

3. Egress host physical nic - vmnic1 (DVUplink1)

 

Thanks for reading

Kev Barrass