VMware Cloud Community
HendersonD
Hot Shot
Hot Shot

Jumbo Frames, VMkernel interface?

I have been working my way through the steps to enable Jumbo Frames for my IP based storage which uses NFS for all of my VMs. I also have two VMs (Exchange 2007 and SQL) that also use this IP storage via LUNs presented to them by the Microsoft iSCSI initiator. So far I have completed these steps:

1. I have an IBM Bladecenter with two switches devoted to my storage network. I enabled jumbo frames on both switches

2. Enabled jumbo frames on the proper interface on my Netapp FAS3020 filers

3. The switches in the Bladecenter are connected to a Cisco 6509 core switch. Enable jumbo frames on the proper interfaces and port-channels on this switch

4. Changed the MTU size on the existing vSwitch that supports my IP storage with this command:

esxcfg-vswitch -m 9000 vSwitch1

The last step still confuses me so I have not attempted it yet. This is the step that creates a VMkernel interface with the correct MTU. Below is my networking setup.

The command to create the VMkernel interface with the proper MTU is:

esxcfg-vmknic -a -I <ip address> -n <netmask> -m <MTU> <port group name>

Can I just create a VMKernel interface on the existing vSwitch1 with the proper MTU? Do I need to create another vSwitch first and then create a new portgroup and VMKernel interace with the proper MTU? I am nervous that I am going to lose connection to my storage and not be able to reconnect.

0 Kudos
7 Replies
AndreTheGiant
Immortal
Immortal

For existing vSwitches you can simple change the MTU.

For vmkernel interface you have to build new interfaces with the right MTU.

Andre

Andrew | http://about.me/amauro | http://vinfrastructure.it/ | @Andrea_Mauro
RParker
Immortal
Immortal

> Can I just create a VMKernel interface on the existing vSwitch1 with the proper MTU? Do I need to create another vSwitch first and then create a new portgroup and VMKernel interace with the proper MTU?

Yes, just add vkernel to existing switch and change the settings, you are done.

0 Kudos
RParker
Immortal
Immortal

> build new interfaces with the right MTU.

For existing interfaces, MTU can be changed, I do it all the time...

0 Kudos
HendersonD
Hot Shot
Hot Shot

As you can see by my networking diagram I already have a vKernel portgroup called NFS that of course has the proper IP address and mask on it. Do I need to delete this portgroup in order to add another one to the same vSwitch if I want to use the same IP address? Will I need to then readd my storage? I am looking for some detail here so I do not lose connection to my storage.

0 Kudos
tWiZzLeR
Enthusiast
Enthusiast

From VMware's iSCSI SAN Configuration Guide, see Enabling Jumbo Frames for Software iSCSI on p.34-35:

Enabling Jumbo Frames for Software iSCSI

Jumbo Frames allow ESX/ESXi to send larger frames out onto the physical network. The network must support

Jumbo Frames end-to-end for Jumbo Frames to be effective. Jumbo Frames up to 9kB (9000 Bytes) are

supported.

Before enabling Jumbo Frames, check with your hardware vendor to ensure your physical network adapter

and iSCSI storage support Jumbo Frames.

Jumbo Frames must be enabled for each vSwitch through the vSphere CLI. Also, if you use an ESX host, you

must create a VMkernel network interface enabled with Jumbo Frames.

Create a Jumbo Frames-Enabled vSwitch

Configure a vSwitch for Jumbo Frames by changing the MTU size for that vSwitch.

Procedure

1 To set the MTU size for the vSwitch, run the vicfg-vswitch -m <MTU> <vSwitch> command from the

vSphere CLI.

This command sets the MTU for all uplinks on that vSwitch. The MTU size should be set to the largest

MTU size among all the virtual network adapters connected to the vSwitch.

2 Run the vicfg-vswitch -l command to display a list of vSwitches on the host, and check that the

configuration of the vSwitch is correct.

Create a Jumbo Frames-Enabled VMkernel Interface

If you are using ESX host, you must use the command-line interface to create a VMkernel network interface

that is enabled with Jumbo Frames.

Procedure

1 Log in directly to the console of the ESX host.

2 Use the esxcfg-vmknic command to create a VMkernel connection with Jumbo Frame support.

esxcfg-vmknic -a -I <ip address> -n <netmask> -m <MTU> <port group name>

3 Run the esxcfg-vmknic -l command to display a list of VMkernel interfaces and check that the

configuration of the Jumbo Frame-enabled interface is correct.

4 Check that the VMkernel interface is connected to a vSwitch with Jumbo Frames enabled.

5 Configure all physical switches and any physical or virtual machines to which this VMkernel interface

connects to support Jumbo Frames.

HendersonD
Hot Shot
Hot Shot

I appreciate everybody's help on this but let me rephrase my question. If you look at my network diagram I have vSwitch1 which already has a Vkernel port attached for my NFS datastore. This is my guess on how to proceed, let me know if I have this correct.

1. I have three ESX servers, put one of them in maintenance mode which will migrate off all the VMs

2. Remove the NFS VKernel portgroup using the vSphere client GUI. This is the reason to put the ESX host in maintenance mode, as soon as the Vkernel portgroup is removed, this ESX server loses connectivity with my centralized storage.

3. From the command line recreate the portgroup of type VKernel using this command:

esxcfg-vmknic -a -I 10.121.126.108 -n 255.255.255.0 -m 9000 "NFS"

Even with the above mentioned method outlined, two big questions I have:

1. What about the VM Port Group I have attached to vSwitch1?

2. What about vmnic5 and vmnic4?

0 Kudos
HendersonD
Hot Shot
Hot Shot

I enabled Jumbo Frames on my IP storage network on one of my ESX hosts. I did this by:

1. Placing it in maintenance mode

2. Deleting shared storage for this host

3. Removing the VMkernel port used for IP Storage via the GUI

4. Recreated the port group and VMkernel port via the command line and made sure that my MTU setting was 9000

5. Added back shared storage which in my case is NFS

6. Took ESX server out of maintenance mode

I wanted to enable Jumbo Frames on my other two ESX servers in the same

fashion. I first tested vMotion and it did not work to move a VM to the

ESX server that had Jumbo Frames enabled. This generated an error so I started another thread:

http://communities.vmware.com/message/1450001

0 Kudos