vDS vkernel interface command line configuration

[fduranti]

Anyone know how to configure a vmkernel port completely from the command line from vMA appliance ?

I want to create some vkernel interface with MTU of 9000 on our distributed vswitch that go to the storage vlan but it seems that I cannot create it. The only way I can create the interface is to:

1) create the vmknic on the Virtual Center from the network configuration of the host.

2) check the port that the vmknic is using with :

esxcfg-vmknic -l

Interface Port Group/DVPort IP Family IP Address Netmask Broadcast MAC Address MTU TSO MSS Enabled Type

vmk3 792 IPv4 10.205.6.211 255.255.255.0 10.205.6.255 00:50:56:76:1d:1c 9000 65535 true STATIC

3) delete the vmknic with the command :

esxcfg-vmknic -d -s dvS-Storage -v792

4) create the new port with the 9000 mtu with:

esxcfg-vmknic -a --ip 10.205.6.211 -n 255.255.255.0 -m 9000 -s dvS-Storage -v 792

running the add command without first creating the vmknic interface from the vCenter result in this error:

# esxcfg-vmknic -a --ip 10.205.6.211 -n 255.255.255.0 -m 9000 -s dvS-Storage -v 793

Valid portgroup/DVPort parameters required.

The only difference is that if I create the interface from vCenter and delete it with esxcfg-vmknic I see that on the vswitch the port is still connected to the dvswitch:

# esxcfg-vswitch -l

DVS Name Num Ports Used Ports Configured Ports Uplinks

dvS-Storage 256 7 256 vmnic6,vmnic4

DVPort ID In Use Client

...

792 0

Probably I'm missing a esxcfg-vswitch command to link the dvportid to the dvswitch but I've tried every kind of command without any result... I don't know if it can be a bug of the esxcfg-vswitch command because also trying to get cdp status get this result:

# esxcfg-vswitch -b dvS-Storage

No such virtual switch: dvS-Storage

Francesco

[lamw]

Have you taken a look at Scott Lowe's post on configuring jumbo frames on VMkernel connect to DVS? http://blog.scottlowe.org/2009/05/21/vmware-vsphere-vds-vmkernel-ports-and-jumbo-frames/

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at:

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

[fduranti]

Yes lawm. I've saw and started with that procedure the problem is that I have currently 12 esx/esxi to configure with 3 different VLAN/vmkernel interface and with that procedure I've to manually configure the vmkernel first on each esx host and then remove and readd them with 9000 mtu...

Also it seems that there's a problem with that procedure using vMA (I was creating a script to automatically create all the vmkernel interfaces at once on all hosts) and this make also impossible to create a vmkernel port with 9000 mtu on the ESXi).

This is what happen on the vMA appliance if I run the 2 commands to delete and recreate the vmkernel nic...

[http://itromvm11.q8int.com|http://itromvm11.q8int.com]$ esxcfg-vmknic -d -v 794 -s dvS-Storage

Removed the VMkernel NIC successfully

[http://itromvm11.q8int.com|http://itromvm11.q8int.com]$ esxcfg-vmknic -a --ip 10.205.6.211 -n 255.255.255.0 -m 9000 -s dvS-Storage -v 793

Can not specify dvsName, dvportId parameters for --add operation.

[lamw]

If you can do it on one host, then this can easily be automated across 20,30 or 100 hosts and vMA is definitely the choice to do this.

One thing I noticed, not sure if it's a typo, but the commands you have listed show different dvPortId, per Scott's instructions it should be the same, so if you delete your dvPortGroup mapped to 794 then you should re-add using that same dvPortId.

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at:

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

[fduranti]

I just copy/pasted 2 command from 2 different test i had run, I forgot to copy the "delete" command in the post so I recreated the vmknic interface and it got the next port (it seems that it continue to get a new port every time I delete/recreate it from the vCenter (I hope I'll not finish ports during those tests).

It seems that on the service console the command run without problem (but the port need to be created from the vCenter graphically) but from vMA the vmknic can be deleted but not recreated... So at this point the problem seems to be 3:

1) I cannot automatically create the vmkernel on dvswitch because I have first to create them manually with vCenter client

2) I cannot create 9000 mtu vmkernel because the esxcfg-vmknic (or vicfg-vmknic) seems to not work on dvswitch from vMA and I cannot do it from ESXi because there's no service console...

Anyone know if there's a way to link a dvPort to a dvSwitch for a server from the command line?

[fduranti]

Anyone know if it's possible to do this configuration manually without VC interaction?

[howie]

you need VC for any vDS operation as that is where most states are saved.

[fduranti]

It's ok that I need the VC to do most of the operation on the vswitch but also using the vMA and connecting through VC doesn't work.

Tthe problem is that mtu on vmkernel can be only configured from command line with esxcfg-vmknic directly from the esx host and only after having created the vmkernel interface from VC.

It don't work from vMA and in this way it's not possible to configure the MTU manually on ESXi or at least I was not able to configure it in any way.

Many commands that work on vswitch work also on dvswitch but with some glitches:

1) on esx host esxcfg-vmknic will let me create a vmkernel interface butonly if I first delete one (because he will not remove the link from the dvport).

2) on vMA (passing through VC or going directly with command on hosts) I can delete a vmkernel vnic on a dvswitch with esxcfg-vmknic -d but I cannot create the vmkernel because the delete command remove the dvport from the dvswitch)

There's an official way to configure mtu of 9000 on ESXi? It's just a bug so I can end to try to configure the ports and wait for a patch ?

Any official word on this can be useful

[lambeth]

I think you can use the following as a workaround for now if this is not your management network (please do file a bug though on lack of MTU in UI):

1. create a standard vswitch on one of the hosts in the vDS, don't put any pNICs on it

2. In the host->config->networking view choose "Distributed Virtual Switch"

3. choose "Manage Virtual Adapters..."

4. select the vmkN you want to change MTU

5. click "Migrate to virtual switch"

6. use your favorite method to change the MTU now that the vmknic is on the standard vswitch

7. click "Manage Virtual Adapters" again

8. click "Add"

9. choose "Migrate existing virtual adapters"

10. in the "Portgroup" column of the dialog, select your old dvPortgroup that the vmknic was originally connected to before you started from the dropdown

11. click "Next" and "Finish"

If the vmknic you want to set the MTU on is also your management network, you would need to put a pNIC on the vSwitch in step 1.

Once you have the MTU confiured, I think (but have not verified) that host profiles would carry it forward. If not please file a bug on that too.

[fduranti]

Hi Lambeth, thanks for your "tips"... it works fantastically. I've done some tests on a machine and I've created via command line a vmkernel interface on a esxi 4.0 host with mtu 9000. When moved via vcenter server the interface mantain the 9000 mtu. If I create an host template, delete the interface and reapply the template the interface is created with an MTU of 1500. Checking inside the data of the template there's no indication of the MTU of the vmkernel interface in any part of it. For us it's not a problem but probably it's a bug that would be nice to get fixed

We've just finished to run a script on vMA that created 4 vmkernel interface on 12 hosts with 9000 mtu without any problems. Tomorrow we will migrate the vlan from the vCenter server (a bit of work but probably less then having to put in mainteinance mode all the hosts to apply the template).

Thanks again for the god tips!!!

[mdri]

I've just begun setting up DVS after installing 4.0U1 and was running into the same problems with MTU. This thread was very helpful, both in introducing me to DVS command syntax and confirming the problems I've had with ESXi 4.0 U1. I could not locate any VMware doc that discusses how to set jumbo frames on DVS virtual interfaces such as a vmkernel port connected to iSCSI storage.

Since the process for configuring Jumbo Frames when using ESX4.0 U1 is very similar to ESX 3.5, I will proceed with implementing DVS.

Unfortunately, the elaborate work around to configure the same in ESXi 4.0U1 is prohibitive. Absent VMware documentation or KB articles, I'll abandon ESXi until VMware makes it easy to implement Jumbo Frames end to end. The UI addition for DVI switch setup in 4.0 was a nice first step but is incomplete. Maybe ESX/i 4.0 Update 2?