So I ran across a very interesting issue (bug) that I was wondering if anyone else can assist with and maybe comment on.
I have several hosts and was suggested by the SAN vendor that Jumbo frames will certainly increase performance (this is a different subject so let's not go down that rabbit hole).
So wanting to do everything the SAN vendor Tech support wanted done...I configured everything as they recommended.
Made sure all the firmware and software drivers were at their respective release levels but simply couldn't get jumbo frames to work.
vmkping -I vmkxx -d -s 8972 ip.add.res.s was failing with sendto() failed (Message too long)
VMware Tech support even said "You have everything setup correctly - it's a hardware vendor issue contact them"
Now the interesting thing was I had two exactly the same hosts - one would work - one would not
Also had another host and it wasn't working with jumbo frames.
So...hours and hours later - support calls with SAN vendor, VMware, HP (servers), Cisco (switch).
Everyone throwing their hands up saying - "Everything looks correct".
I carefully compared working host to non-working host
It would appear that the order that you define the vmkernel ports is VERY important (news to me).
See below table of NON-working configuration
Simply switching the order of the vmk got everything working (both in 5.1 and 5.5)
So the first vmk port defined for the subnet had jumbo frames enabled got everything working.
Interested in your comments or if you have ran across this issue before
Host1 - NOT working
vmk0 10.0.100.80 mtu 1500
vmk1 172.31.1.190 mtu 1500
vmk2 172.31.1.90 mtu 9000
vmk3 172.31.1.100 mtu 9000
vmk4 172.31.1.80 mtu 1500
vmk5 172.31.1.180 mtu 1500
Host2- IS working
vmk0 10.0.100.81 mtu 1500
vmk1 172.31.1.91 mtu 9000
vmk2 172.31.1.81 mtu 1500
vmk3 172.31.1.191 mtu 1500
vmk4 172.31.1.101 mtu 9000
vmk5 172.31.1.181 mtu 1500
Host3- NOT working
vmk0 10.0.100.95 mtu 1500
vmk1 172.31.1.195 mtu 1500
vmk2 172.31.1.85 mtu 1500
vmk3 172.31.1.95 mtu 9000
vmk4 172.31.1.105 mtu 9000
vmk5 172.31.1.185 mtu 1500
