Extremely slow network performance in VMs with 10GbE host

[i0ntempest]

Hi all,

I don't remember when this problem started (at least a few months) but I am sure it wasn't like this before. My guests have been getting extremely low download speeds recently (<5Mbps) from the internet, while the upload speed is completely unaffected (800+ Mbps on a gigabit service). Which is extremely weird.

My hosts, both running macOS (12.4/12.5 beta), have Aquantia/Marvell AQC107 cards installed, one is a PCIe AIC and another one is a Thunderbolt dongle. This problem occurs on both of them in all VMs, regardless on what OS they're running. 

During troubleshooting I did the following:

On the host, if I opt to use the builtin gigabit interface, the download speed in the VM would be around 200 Mbps. Not full speed, but acceptable.

When host is using 10GbE with 9000 MTU, setting the guest to also use 9000 MTU would increase download speed to around 300 Mbps. Again not full speed but acceptable.

Changing MTU back to 1500 on the host alone won't do anything.

Tried resetting virtual network adapters with vmnet-cli command, didn't change anything.

Anyone have any ideas? Thanks in advance.

[continuum]

You forgot to mention the 3 most important parameters ...
- virtual hardware ? - if you use AMD pcnet32 everything you say is normal
- bridged / hostOnly / NAT / guestonly ???
- guestOS ?

[i0ntempest]

Thanks for the reply.

I've tried e1000e, e1000, and vmxnet3 virtual network cards, all of them has this problem. I'm using bridged networking so that VMs can grab IP directly off my router, NAT ("shared with my Mac") does not have this issue. As for the guest OS, I tried Linux and Windows and both have the issue,

[Technogeezer]

The AQC107 is a PCIe card. What do you mean by "Thunderbolt dongle"?

Have you ruled out any speed autonegotiation issues at your switch?

Are each of these NICs on separate subnets?

Have you run networking diagnostic tools such as traceroute in an attempt to get an idea where the bottleneck is happening?

Have you measured throughput from the host standpoint not in a VM (and not "downloading" a file, but actual raw throughput through the network interface, as "downloading" introduces the speed of writing a file to disk into the equation).

For completeness in getting all information we can, what Mac model are you running this configuration on?

What NIC driver are you using on the Mac, assuming this is not an Apple NIC. Marvell does not have a download for a driver for this card. A quick search of the 'net says that while this card may have worked with older macOS releases and the Apple NIC driver, Big Sur and Monterey required a "patch/hack" to the Apple driver to get it to work. Which if that's the case, this issue may fall on deaf ears with VMware since it's not a configuration supported by Apple or the vendor.

[i0ntempest]

There are AQC107 based Thunderbolt ethernet adapters, and I'm using one of them (Promise SANLink3 N1) with my Mac mini. There are driver for this chipset in the OS, in fact Mac Pro uses this exact chipset for 10GbE. The PCIe card does need a kext patch starting from Big Sur, but the Thunderbolt adapter works natively in macOS and does not need any patches.

I can rule out autonegotiation issues on the switch, it's flashing a greed light indicating a 10G link.

All of my devices are on the same subnet.

traceroute does not indicate any problems. The ping is fine, just the speed.

My hosts can reach a download speed of 1.2 Gbps from the internet. I tested with iperf as well, using reverse mode (guest is downloading) and host as the server, the speed is around 50 Kbps. In this test no traffic was going through my switch or router, just direct communication between host and client.

One of my hosts is a i9-9900K hackintosh (with AQC107 PCIe card), another is a 2018 Mac mini (with the Thunderbolt adapter). I understand if you/VMware doesn't want to support hackintoshes, but the issue is present on a real Mac as well.

[Technogeezer]

Thanks. Although I don’t have any thoughts right now on this, others may find the info useful to complete the view of what’s going on. 

[ColoradoMarmot]

Definitely no support or discussion allowed if violating Apples EULA.

For the real host, there could be a couple of things going on - Fusion 12 uses the native networking stack, so it's possible there's an issue on the MacOS side, rather than fusion.  In any case, as I recall, Fusion doesn't provide more than 1G support for networking to guests.

[i0ntempest]

Thanks for the info. As I recall this indeed started after an OS upgrade, though I can't remember which version.