twork
Contributor
Contributor

Linux ethernet bridging between virtual switches

Jump to solution

As a test for a larger project, I'm trying to use a Linux VM as a bridge between two virtual networks on a single ESX 3 machine. The bridge setup within Linux isn't complicated, and the fact that this isn't working makes me wonder if the problem is related to my VMware system.

Setup details: Three Linux VMs and two virtual switches. Rough diagram:

[VM 1]
eth0: IP 10.1.1.16
|
[virtual switch 1]
|
eth1: no IP
[Bridge VM]
eth2: no IP
|
[virtual switch 2]
|
eth0: IP 10.1.1.17
[VM 2]

"Bridge VM" is a Linux VM with interfaces eth1 and eth2 bound to a bridge device. So, if everything goes according to plan, "VM 1" and "VM 2" should be able to see each other as if they were on the same ethernet segment.

But, for example, when VM 1 tries to ping VM 2, I can see ARP who-has requests crossing the bridge machine, and the bridge device correctly learns the MAC addresses of both endpoint interfaces, but ARP replies aren't propagated back across the bridge to the first host, and the pings time out.

In the process of trying to track down the problem, I've found that if I bring both "VM 1" and "VM 2" up on the same virtual switch -- say, "virtual switch 2" in the diagram -- all three machines can see each other. If I then assign an address to eth2 on the bridge machine, VM1 and VM2 can ping it. Same applies on switch 1, so I know that both virtual networking instances work.

But, if I put things back the way they are in the diagram, switch off bridging on the middle machine, and assign IP addresses to each of its NICs, networking will only work between the middle system and whichever VM comes on line first. If VM 1 is pinging away at eth1 on the middle machine, VM 2 can't see the network at all, and attempts to ping come back "host unreachable". If I shut off VM 1, or just disconnect its NIC from virtual switch 1, after a few seconds VM 2's networking comes back up again, on virtual switch 2.

So, it appears that "virtual switches", at least in my setup, aren't really very well virtualized. Bug? Or, is there some setting I haven't got right?

Possibly relevant: I haven't changed any "VLAN" settings from the default set when virtual networks are created. I don't know if that matters or not.

Also possibly relevant, and weird: Each "endpoint" machine only has the one NIC, but the bridge has three; in addition to the two used for attempted bridging, it has eth0 attached to a (third) virtual switch which is connected to a physical NIC on the ESX host. Networking on that device has never had a problem, and as far as I can tell, hasn't interfered with networking on either of the two "entirely virtual" switches, the way they appear to interfere with each other.

I'm stumped. Clues for where to gather more information are welcome...

--Michael

0 Kudos
1 Solution

Accepted Solutions
WillemB
Enthusiast
Enthusiast

I think your network security settings are problematic. You can enable

-> Promiscuous mode (default reject)

-> Forged transmits (default allow)

-> Mac address change (default allow)

I think you might need promiscuous mode to allow bridging (proxying the raw network packages)

View solution in original post

0 Kudos
3 Replies
WillemB
Enthusiast
Enthusiast

I think your network security settings are problematic. You can enable

-> Promiscuous mode (default reject)

-> Forged transmits (default allow)

-> Mac address change (default allow)

I think you might need promiscuous mode to allow bridging (proxying the raw network packages)

View solution in original post

0 Kudos
twork
Contributor
Contributor

Yep, that did the trick. Thanks.

0 Kudos
RenaudL
Hot Shot
Hot Shot

I haven't looked at your setup in details, but be aware that bridging 2 virtual switches with a VM may loop your network. Please carefully check your setup and whether you really need it before designing anything on top of it.

0 Kudos