Enabling Jumbo Frames on an existing environment.

[nathanw]

Hi, I have a VMware ESX 3.5 environment that I am migrating/upgrading to vSphere 5.

After upgrading the Hosts I will be implementing Jumbo Frames for the iSCSI Storage.

The environment consists of the following;

Dell/EMC AX4-5i SAN;

3 x ESX Hosts using ESX software iSCSI;

2 x Windows Hosts using windows 2k8 R2 iSCSI initiator;

Cisco 3750 switch stack.

My question is;

To minimise disruption to the environment how would I best proceed in enabling Jumbo Frames end to end?

Is there disruption to the Array when enabling Jumbo Frames?

I can evacuate the ESX Hosts prior to changing the iSCSI port Groups.

My thoughts are;

Enable Jumbo Frames on the Switch ports for ESX Servers and SAN;

Enable Jumbo Frames on the ESX Hosts and Windows hosts

Enable Jumbo Frames on the SAN

Your thoughts and experiences would be much appreciated, I am used to Fibre storage.

[proden20]

-----Enabling jumbo frames on your AX4-5, and potential issues:

http://knowledgebase.emc.com/emcice/documentDisplay.do;jsessionid=F041572B39434C52DB1A5D3FB479D5F3?d...

"Note: If you change the MTU for PORT 0 to 9000 (as, for example, in a  CX3-40c), due to the array design, it will also cause every initiator  (host) on PORT 1 to log out. Also, any host NICs still running at 1500  will still log in to the array port set to 9000, albeit only at 1500.   The setting simply states that the array will negotiate an MTU setting  up to 9000 as long as everything in between can also negotiate the  setting."

This sounds to me like once you've configured one SP port for MTU 9000, that the array will shut down any ports not set to 9000 until you fix that, perhaps someone else can interpret it differently.  I'd plan on a brief interruption and have all your writes completed and I/O quiesced to be sure.

-----Enabling jumbo frames at the vSwitch / vmKernel:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=100765...

I think your plan makes sense...

[nathanw]

Proden20 Thanks for this information. Very helpful but I can't access that EMC document.

[rickardnobel]

Nathan wrote:

I can evacuate the ESX Hosts prior to changing the iSCSI port Groups.

My thoughts are;

Enable Jumbo Frames on the Switch ports for ESX Servers and SAN;

Enable Jumbo Frames on the ESX Hosts and Windows hosts

Enable Jumbo Frames on the SAN

It might be good to do a reboot of the host after the setup is complete. This is to make sure that a new TCP session is setup with the iSCSI SAN. It is only through the three-way handshake that the two parts will decide if to use jumbo frames or not.