VMware Cloud Community
ian4563
Enthusiast
Enthusiast

Anyone play with jumbo frames yet?

On my newly upgraded 3.5 cluster I setup jumbo frames per the instructions on page 63 of the server config guide. I have tried MTU's of 1500, 4500, and 9000 and see no difference in vmotion throughput between any of these. The vmotion VLAN is set to allow jumbo frames on my HP switch and I found a pdf on VMWare's website the says they support jumbo frames on my Intel 82546 Intel NIC. Am I expecting the wrong thing out of jumbo frames?

0 Kudos
3 Replies
tsightler
Hot Shot
Hot Shot

I can't imagine that jumbo frames would have much impact on vMotion performance. Were you having bad performance with vMotion previously? With ESX 3.02 without jumbo frames I can vMotion a 4GB virtual machine in about 50 seconds. The theoretical maximum speed for transferring 4GB on a 1Gb link is about 120MB/s and you're unlikely to see more than about 100MB/s in the real world, jumbo frames or not. That means in the absolute best case it would take 35 seconds just to transfer the data and that does not include the setup time, quiesce time, etc so perhaps, assuming jumbo frames helped the throughput, I might be able to get the vMotion of my 4GB system done in 40 seconds rather than 50. Based on other factors it's hard to know if you would even get that much difference.

I think it's far more likely that you would see benefit from jumbo frames if you have virtual machines which make heavy use of network bandwidth. For example, we're seeing some improvement from VM's that use software iSCSI initiators within the virtual machine, not a huge boost in overall throughput, but a minor boost while actually lowering the overall CPU usage of the host which is still a nice win overall. I could also see jumbo frames helping on 10Gb links by lowering the the number of interrups which should lower the overall CPU impact of such high speeds.

Later,

Tom

0 Kudos
carl_henderson
Contributor
Contributor

depending on the type of data, jumbo frames can actualy slow down performance.

you will probably only see the benefit of jumbo frames when using database traffic, iSCSI or other large data applications

0 Kudos
ian4563
Enthusiast
Enthusiast

Thanks guys. I'm not a networking expert and I guess I was not expecting the right things out of jumbo frames. It just seemed to me that there would be less TCP overhead and thus better throughput and CPU usage, but you're right it's not like it takes that long to vmotion anyway.

0 Kudos