VMware Networking Community
scale21
Enthusiast
Enthusiast

NSX VXLAN Bridge questions

Is it possible to setup a vlxan to vlan bridge so that your VXLAN nsx backed machines can communicate with a portgroup on the same VDS that has a standard portgroup created with say vlan10?

I have to assume the answer is yes.

When i setup my bridge i setup a DLR with a management interface. When i go to the bridge section and try it add my logical switch with my vms on it, it doesnt show up?  Can you only have one edge device / dlr per logical switch? That doesnt seem correct in that edge gateways and routers are different functions.

I have well over 40 logical switches but when trying to select any of them via the bridging button they dont show up. NSX 6.2.5.

I must be missing something obvious.

**EDIT. I got the bridge setup but there is no communication. I left the interface of the DLR on on my mangagement network.  WOuld i also need to add an interface for the network i am communicating on? That almost makes no sense as this is a bridge at L2 and not a route to a different network. My initial thought is it should broadcast, find the other machine on the same l2 network and communicate without anything extra.

I cant communicate either direction. My VLAN backed machine cant get to my other vxlan backed machines or the edge services gateway (default gateway). My VXLAN machines cant talk to my machines in my VLAN. Not sure what i am missing here. I have tried this in 2 different labs. No dice. I get stuck at this exact point. Both machines are on the same host so i wouldnt think it could be an upstream switch.

Reply
0 Kudos
4 Replies
cnrz
Expert
Expert

Vxlan-Vlan bridging allows VMs on LVxlan based ogical Switch and VMs on Vlan based Port group to communicate. Instead of port group based VM it could be a Physical host on that Vlan. Since the technology is bridging, both sides should be on the same IP subnet since there is no router in between. What are the IP subnets of VMs on Vxlan and Vlan Port groups?

Due to Spanning-Tree there is only one host chosen as the  "bridge-point" to prevent loops, and this host is where the active DR Control VM is located. It could be found as below: (even if both of the VMs are on the same ESX host, they first need to go to this bridge host, so it may be useful to check

  1. Both VMs on same IP subnet
  2. first if the bridging is working on that host Vmotioning the VMs to this specific ESX where the active DLR Control VM is located. (It could be found as below Manage>Settings>Configuration of the DLR)
  3. Logical Swithing and DLR needs to ensured to be working (even for VMs on the same host that DLR Control VM is not hosted on), so first 2 VMs on the same port-group  (1 is on ESXx that DLR Control VM is on, the other is on ESXy) can ping? Similarly the VMs while remaining on these 2 hosts, this time taken to Logical Switch can ping?
  4. The Mac Address table of the bridge should see both VM on Logical Switch and VM on Vlan based Port group in its Mac address table. Are both of the Mac addresses on the mac-address table?

bridge_which_host.png

The traffic flows similar to this path:

http://www.routetocloud.com/2014/10/nsx-l2-bridging/

bridge-traffic_flow.png

These links may be helpful how to see the mac address table of the bridge:

http://www.yet.org/2014/09/nsxv-troubleshooting/

Bridging

To get more information on a all bridge instance hosted on your logical router

# show control-cluster logical-routers bridges <lr-id> all
LR-Id           Bridge-Id   Host                 Active
1460487509        1          192.168.110.52       true

And now the Mac address on them

# show control-cluster logical-routers bridge-mac <lr-id> all
LR-Id       Bridge-Id   Mac               Vlan-Id Vxlan-Id Port-Id   Source
1460487509  1           00:50:56:ae:9b:be 0       5000     50331650  vxlan

http://www.routetocloud.com/2014/10/nsx-l2-bridging/

net-dvr_bridge.png

For configuration of bridging:

http://wahlnetwork.com/2014/11/17/working-nsx-layer-2-bridging/

http://buildvirtual.net/vcp-nv-configure-and-manage-layer-2-bridging/

Reply
0 Kudos
scale21
Enthusiast
Enthusiast

I am not sure i follow what you are saying. I have seen this webpage of the graphics shown but it doesnt explain the setup on the Bridge DLR itself.

here is my configuration i am trying to get to bridge.

VXLAN VNI 5001 - Logical Switch: VxLAN5001

VLAN ID 10  - Distributed Port Group: VLAN10

VLXAN virtual machine: 172.16.1.1

VLAN virtual machine 172.16.1.2

Both vms are on the same host. No physical switch.

The bridge is configured between the logicial switch that houses VNI 5001 and the VDS portgroup that houses VLAN 10.

The machines cannot ping each other. Destination unreachable.

The mac addresses dont show up in the arp tables of each other.

IF i look at the Distributed port group and its ports i do see my VM in here connected with state "LINK UP" and green

I also see the the DLR in here with a LINK UP  green status

If i look at the logical switch, i can see my DLR in here showing LINK UP and green as well.

If i login to the console of the DLR itself and attempt a ping from it to either VM i also get "Network is Unreachable"

I am running out of ideas.

Reply
0 Kudos
scale21
Enthusiast
Enthusiast

I got communication from my VXLAN to VLAN now with ping but i cant ping back the other direction.  IF i look on the source machine in VxLAN i do see an arp entry for my remote vlan machine but i cannot ping or browse. I have checked that there is no windows firewall or other things enabled on the source and destination machine.

Any ideas?

Reply
0 Kudos
cnrz
Expert
Expert

Are vlan port group and Vxlan port group connected to the same Distributed switch? (Should be)

The bridge Mac address table shows both Vlan and Vxlan VMs Mac addresses?

Is it possible to check the firewall rules (Edge and dFw) to ensure there is no firewall rule blocking ICMP?

If the places of the VMs changed, i.e. Vxlan VM is put to Vlan port group, and the Vlan VM is put to Vxlan logical switch, does the result change?

For more detailed packet flow, Flow Monitor>Live Flow could be used for both VMs if the packets are reaching to VM or not. One other option could be packet capture on the Vnic of the Vxlan VM.

This link on more detailed packet flow explains:

L2 bridge | Chan's Blog

Reply
0 Kudos