mgmist
Contributor
Contributor

Upgrading to MacOS Big Sur broke (packet corruption) network bridge connected to VLAN interface

Greetings! Came here looking for a solution/workaround but I didn't find one.

My diagnosis is that MacOS Big Sur has a problem but I am not sure how to report it to Apple.

Edit: I reported to Apple: https://discussions.apple.com/thread/252999905

Posting it here in case my diagnosis is wrong and the problem is rooted in VMware Fusion's use of MacOS.

Problem summary: ARP reply send by Juniper switch arrives on Ethernet physical port "en9" with 802.1Q tag (VLAN ID 44). Packet is seen tagged on parent interface "en9" and untagged on child VLAN interface "vlan1". Interface "vlan1" is a member of bridge interface "bridge100" (created by VMware Fusion). Packet is seen corrupted (no checksum error) on "bridge100".

Workaround: Move VLAN interface from host to guest (Debian 10). (This also suggests that bridge in MacOS has the problem).

Instead of

[host en9 --- vlan1 --- bridge100 --- en6] --- [guest ens37]

do

[host en9 -- bridge100 --- en6] --- [guest ens37 --- ens37.44] 
Here, too, bridge100 and en6 are created by VMware Fusion. I created ens37.44 manually.

My question: what is the correct procedure to report this issue to Apple or VMware? Is there a fix?

Best,

-Manjiri Gadagkar, Juniper Networks.

 

Details: GuestOS interface MAC & IP : 00:50:56:2b:a8:46 & 11.20.4.3

 

 

ifconfig en9
en9: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
	options=404<VLAN_MTU,CHANNEL_IO>
	ether b8:8d:12:54:84:7f 
	inet6 fe80::104b:4530:f829:257e%en9 prefixlen 64 secured scopeid 0x5 
	inet 10.0.0.19 netmask 0xffffff00 broadcast 10.0.0.255
	nd6 options=201<PERFORMNUD,DAD>
	media: autoselect (100baseTX <full-duplex,flow-control>)
	status: active

sudo tcpdump -i en9 -exxn vlan 44 and arp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on en9, link-type EN10MB (Ethernet), capture size 262144 bytes
15:27:21.926889 00:50:56:2b:a8:46 > fc:96:43:be:d1:82, ethertype 802.1Q (0x8100), length 64: vlan 44, p 0, ethertype ARP, Request who-has 11.20.4.1 tell 11.20.4.3, length 46
	0x0000:  fc96 43be d182 0050 562b a846 8100 002c
	0x0010:  0806 0001 0800 0604 0001 0050 562b a846
	0x0020:  0b14 0403 0000 0000 0000 0b14 0401 0000
	0x0030:  0000 0000 0000 0000 0000 0000 0000 0000
15:27:21.930241 fc:96:43:be:d1:82 > 00:50:56:2b:a8:46, ethertype 802.1Q (0x8100), length 64: vlan 44, p 0, ethertype ARP, Reply 11.20.4.1 is-at fc:96:43:be:d1:82, length 46
	0x0000:  0050 562b a846 fc96 43be d182 8100 002c
	0x0010:  0806 0001 0800 0604 0002 fc96 43be d182
	0x0020:  0b14 0401 0050 562b a846 0b14 0403 0000

 

 

 

Above is as expected. 

 

 

ifconfig vlan1
vlan1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        ether b8:8d:12:54:84:7f
        inet6 fe80::877:2c42:4c21:9c61%vlan1 prefixlen 64 secured scopeid 0xf
        nd6 options=201<PERFORMNUD,DAD>
        vlan: 44 parent interface: en9
        media: autoselect (100baseTX <full-duplex,flow-control>)
        status: active
sudo tcpdump -i vlan1 -exxn arp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vlan1, link-type EN10MB (Ethernet), capture size 262144 bytes
13:12:54.306670 00:50:56:2b:a8:46 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 11.20.4.1 tel\
l 11.20.4.3, length 46
        0x0000:  ffff ffff ffff 0050 562b a846 0806 0001
        0x0010:  0800 0604 0001 0050 562b a846 0b14 0403
        0x0020:  0000 0000 0000 0b14 0401 0000 0000 0000
        0x0030:  0000 0000 0000 0000 0000 0000
13:12:54.312213 fc:96:43:be:d1:82 > 00:50:56:2b:a8:46, ethertype ARP (0x0806), length 60: Reply 11.20.4.1 is-at fc:96:4\
3:be:d1:82, length 46
        0x0000:  0050 562b a846 fc96 43be d182 0806 0001
        0x0010:  0800 0604 0002 fc96 43be d182 0b14 0401
        0x0020:  0050 562b a846 0b14 0403 0000 0000 0000
        0x0030:  0000 0000 0000 0000 0000 0000

 

 

 

Above is as expected.

 

 

 

ifconfig bridge100
bridge100: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        options=3<RXCSUM,TXCSUM>
        ether 16:7d:da:11:a9:64
        Configuration:
                id 0:0:0:0:0:0 priority 0 hellotime 0 fwddelay 0
                maxage 0 holdcnt 0 proto stp maxaddr 100 timeout 1200
                root id 0:0:0:0:0:0 priority 0 ifcost 0 port 0
                ipfilter disabled flags 0x0
        member: vlan1 flags=3<LEARNING,DISCOVER>
                ifmaxaddr 0 port 15 priority 0 path cost 0
        member: en6 flags=3<LEARNING,DISCOVER>
                ifmaxaddr 0 port 21 priority 0 path cost 0
        Address cache:
                fc:96:43:be:d1:82 Vlan1 vlan1 1198 flags=0<>
                0:50:56:2b:a8:46 Vlan1 en6 1198 flags=0<>
        media: autoselect
        status: active


sudo tcpdump -i bridge100 -exxn arp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on bridge100, link-type EN10MB (Ethernet), capture size 262144 bytes
13:14:56.271224 00:50:56:2b:a8:46 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 11.20.4.1 tel\
l 11.20.4.3, length 46
        0x0000:  ffff ffff ffff 0050 562b a846 0806 0001
        0x0010:  0800 0604 0001 0050 562b a846 0b14 0403
        0x0020:  0000 0000 0000 0b14 0401 0000 0000 0000
        0x0030:  0000 0000 0000 0000 0000 0000
13:14:56.275495 fc:96:43:be:d1:82 > 00:50:56:2b:a8:46, ethertype ARP (0x0806), length 64: Unknown Hardware (44) (len 0)\
, ARP (len 1), length 50
        0x0000:  0050 562b a846 fc96 43be d182 0806 002c
        0x0010:  0806 0001 0800 0604 0002 fc96 43be d182
        0x0020:  0b14 0401 0050 562b a846 0b14 0403 0000
        0x0030:  0000 0000 0000 0000 0000 0000 0000 0000

 

 

 

Above is a corrupt packet.

Packet arrived healthy on "vlan1". On "bridge100", bytes at offset 0x000c and 0x00d are 0806 as expected, but the rest of the bytes are mislaid. There are 4 extra bytes, between DMAC and EthType field. The guest does not recognize this ARP reply.

0 Kudos
1 Reply
brianblood
Contributor
Contributor

Manjiri,

Excellent tear-down of the problem many of us have been experiencing.

I would implore you to open a bug-report/Feedback to Apple with all of this information.

One minor correction, the corruption seems to be an extra four bytes after the EthType field.

 

- B

0 Kudos