VMXNET /TCP Segmentation Offload / Jumbo frames

[JaapL]

Hi all,

I 'm building a proof of concept of Vsphere ESXi 4 (with all updates) with an iSCSI (software Starwind) SAN.

I 've read in several post about a possible performance issue when using iSCSI.

Therefore I 'm searching through the communities/docs etc. and came across TSO and Jumbo frames.

If possible could you confirm my next 'statements':

1. For TSO and Jumbo frames you need to use VMXNET enhanced of VMXNET3

2. VMXNET enhanced and VMXNET3 options can be chosen when creating a new VM (also when installing W2K3 standard 32/64 using ESXi v4)

2a. VMXNET needs VMware tools installed before being able to us it (KB Article: 1001805)

2b. VMXNET enhanced and VMXNET3 do not need VMware tools to work when creating a new VM

2c. You need the workaround (mentioned in KB Article: 1007195), to get VMXNET enhanced working in W2K3 standard 32/64

2d. In this KB they failed to mention you have to use the the network settings and use the MAC-address from the removed NIC.

3. TSO is enabled by default (TSO MSS set to 65535) in ESXi V4 > esxcfg-vmknic -l, checked this after a straightforward install.

4. Jumbo frames are not enabled by default in ESXi v4 > vicfg-vswitch -l, but can be enabled without deleting vSwitch(x) and creating a new one.

5. TSO and Jumbo frames can be implemented seperately. ( What is the average performance gain by implementing one or both options?)

Why can't these settings be implemented in the VI client ???

Thanx for your patience.

Jaap

[AndreTheGiant]

1. For TSO and Jumbo frames you need to use VMXNET enhanced or VMXNET3

Right

2. VMXNET enhanced and VMXNET3 options can be chosen when creating a new VM (also when installing W2K3 standard 32/64 using ESXi v4)

There are some tricks to edit the vmx and change the adapter type.

2a. VMXNET needs VMware tools installed before being able to us it (KB Article: 1001805)

2b. VMXNET enhanced and VMXNET3 do not need VMware tools to work when creating a new VM

I haven't notice... cause I always install VMware Tools

3. TSO is enabled by default (TSO MSS set to 65535) in ESXi V4 > esxcfg-vmknic -l, checked this after a straightforward install.

Right

4. Jumbo frames are not enabled by default in ESXi v4 > vicfg-vswitch -l, but can be enabled without deleting vSwitch(x) and creating a new one.

Right

5. TSO and Jumbo frames can be implemented seperately. ( What is the average performance gain by implementing one or both options?)

Have a look at

http://www.vmware.com/files/pdf/ESX_networking_performance.pdf

http://www.vmware.com/files/pdf/perf_comparison_virtual_network_devices_wp.pdf

And also:

Andre

[JaapL]

Hi Andre,

Thanks for your quick response.

ad 2.

Haven;t found the trick yet, but will keep on searching

ad 2a/2b.

Maybe I should have put the question differently:

- You can use VMXnet enhanced/VMXnet3 with new VM install

- You cannot get VMXnet enhanced/VMXnet3 working straightaway

- It will work after boot, installing VMware tools, reboot

ad 5

Found a performance report on VMXnet3: http://www.vmware.com/resources/techresources/10065

Seems to me that VMXnet3, compared to VMXnet enhanced, is especially useful when having a network above 1Gb.

I havent found/read yet what is the best way to combine several 1Gb connections towards a SAN storage.

- Do nothing and let VMware figure it out itself

- NIC teaming

- Round robin

Thanx

Jaap

[AndreTheGiant]

Haven;t found the trick yet, but will keep on searching

See http://malaysiavm.com/blog/how-to-change-virtual-machine-network-adapter-manually/

Seems to me that VMXnet3, compared to VMXnet enhanced, is especially useful when having a network above 1Gb.

Right... but have also a lot of limitation. For example you cannot enable FT on a VM with vmxnet3.

Andre

[JaapL]

Hi Andre,

Thanks for the link how to ....

Will try to implement VMXnet enhanced next week.

Have read your document and link about VMXnet3 and FT > restrictions.

Hope VMware will be able to solve this in the near future.

Thanks,

Jaap