Virtual machine guest drops packets using e1000 adaptor in VST Mode

[john_gallucci]

We discovered a recent issue in our datacenter where our virtual machine guests will drop packets when using Jumbo Frames on the e1000 virtual adapter on a vSwitch that is configured for VST mode.  The Guest operating system is Red Hat Enterprise Linux 5 Update 6.

The dropped packets occur when the MTU is set to 9000 on the Linux OS.  The vSwitch is properly configured for jumbo frames and we tested 2 virtual machines both on the same hardware (and same VM network) as well as two different ESXi servers to verify this.

Our wireshark caputures revealed that the last 4 bytes of the packet payload are dropped.  An example is the send packet on one server was 8142 bytes and the receiving packet was 8138 bytes.  Because of the difference, the TCP checksum fails and the packet is discarded.  Somewhere through the transfer the 802.1q header is missing on one end and present on the other, minus the last 4 bytes at the end of the frame.

So far our only workarounds are to set the MTU to 1500 on the Linux OS or to use the VMXNET3 adapter with MTU 9000.  We use PXE to boot our VMs which is why we have chosen to use the e1000 adapter rather than the VMXNET3 adapter (although RHEL 6 supports PXE with the new adapter, so that is good news in the future!)

[timemachine · Accepted solution]

Hi,

You cannot use e1000 with jmbo frames this is by esx design,Only enhanced vmxnet (and vmxnet3) adapters support jumbo frames.Adapters other than enhanced vmxnet (and vmxnet3) adapters, for example the E1000 adapter, cannot be used with jumbo frames. Attempts to change the MTU appear to succeed but the adapter always drops frames larger than 1500 bytes.
TM

[john_gallucci]

You are correct, thank you for the reply.  We went ahead and changed all of our virtual machines to vmxnet3 and the system behaves normally.  We decided to put eth0 on e1000 for PXE boot and use the other interfaces on vmxnet3 for jumbo frames.