3.5 Install / Notes

[RParker]

I found a few things, for one SQL needs to be configured for Native SQL Client DSN (I had to reconfigure the DSN). Then it will upgrade the database schema.

Then this takes quite a long time to upgrade the database. That's all I found for now.

[11:27a EST - The VIM (Virtual Infrastructure Management) install was fine. Everything looks good for client install (Vista 32-bit)

3.5 Server I upgraded one of my servers, and I had invalid sda (even thought it boots fine with 3.02). I tried upgrading, no good. I am now doing a NEW install (retaining current VMFS configurations). Looks like I will be here a while cleaning up .. but hey, at least I can test 3.5.

[admin]

This certainly feeels like a Gremlin..... I'm adding 3 nics to the first switch SC then setting up Active/Active/Standby <vmnic0,vmnic1,vmnic2> then deleleting vmnic2, the vswitch retains its Active/Active property.

Hope this is of help.;)

[admin]

Below is all the options you have for ESX 3.5 if you want to script it

vswitch_setpolict vswitch

--securepolicy-promisc=bool

Allow promiscuous mode.

--securepolicy-macchange=bool

Allow MAC address to be changed.

--securepolicy-forgedxmit=bool

Allow forged transmits.

--shapingpolicy-enabled=bool

Enabled shaper.

--shapingpolicy-average-bandwidth=int

Average bandwidth.

--shapingpolicy-peak-bandwidth=int

Peak bandwidth.

--shapingpolicy-burst-size=int

Burst size.

--nicorderpolicy-active=vmnic_list

List of active NICs.

--nicorderpolicy-standby=vmnic_list

Order in which failover should occur.

--failurecriteria-check-speed=failurecriteria-check-speed

Detecting failover using link speed check

--failurecriteria-speed=int

The speed for link speed check method

--failurecriteria-check-duplex=bool

Detecting failover using link duplex check

--failurecriteria-duplex=bool

The duplexity for link duplex check method

--failurecriteria-check-error=bool

Detecting failover using link error percentage

--failurecriteria-error=int

The error percentage for link error percentage check method

--failurecriteria-check-beacon=bool

Detecting failover using the beacon

--nicteaming-policy=nicteaming-policy

Nic-Teaming policy.

--nicteaming-reverse-policy=bool

Apply the teaming policy to inbound frames as well.

--nicteaming-notify-switch=bool

Notify switches when detects a link failure.

--nicteaming-rollingorder=bool

Whether or not to use rolling failover.

--offloadcaps-csum-offload=bool

Checksum offload

--offloadcaps-tcp-segment=bool

TCP segment offload

--offloadcaps-zerocopy-xmit=bool

Scatter gather

[ExCon]

Okay, I can confirm that this workaround is working for me, too. I'm building with the UDA, and all I really did was throw your unlinking/linking/nicorder in at the end of my config. So, at the end of my UDA config, I included the following lines for each vSwitch I needed to reconfigure:

/usr/sbin/esxcfg-vswitch -U vmnic0 vSwitch0

/usr/sbin/esxcfg-vswitch -U vmnic1 vSwitch0

/usr/sbin/esxcfg-vswitch vSwitch0 -L vmnic0

/usr/sbin/esxcfg-vswitch vSwitch0 -L vmnic1

service mgmt-vmware restart

sleep 20

vimsh -n -e "hostsvc/net/vswitch_setpolicy --nicorderpolicy-active vmnic0 vSwitch0"

vimsh -n -e "hostsvc/net/vswitch_setpolicy --nicorderpolicy-active vmnic0,vmnic1 vSwitch0"

You rock!

[ExCon]

The rollingorder (now failback?) should also be mentioned here, because it's very confusing.

Under 3.0.x, the following statement set rolling failover to Yes:

vimsh -n -e '/hostsvc/net/vswitch_setpolicy --nicteaming-rollingorder=true vSwitch0'

Under 3.5, the following statement sets failback to No:

vimsh -n -e '/hostsvc/net/vswitch_setpolicy --nicteaming-rollingorder=true vSwitch0'

See the similarity? It's counterintuitive, but 3.0.x rolling failover Yes = 3.5 failback No. But you set them with the same statement and same values!

[dominic7]

I can confirm that it works for 2 vmnics in a vswitch... argh. I can keep running the same process of unlinking and linking and restarting the mgmt-vmware to get one additional vmnic linked each time. In my case I have 4 vmnics in a team so I've got to run that vswitch through 3 iterations of unlinking/linking/restart wait...

This is horribly annoying.

[garybrown]

do you really need the vimsh nicorder command...?

I have simply added the 'U'nlink and 'L'ink commands and finished off with /usr/bin/vimsh -n -e /internalsvc/refresh_network and that seems to work for me

eg

/usr/sbin/esxcfg-vswitch -U vmnic2 vSwitch1

/usr/sbin/esxcfg-vswitch -U vmnic4 vSwitch1

/usr/sbin/esxcfg-vswitch -L vmnic2 vSwitch1

/usr/sbin/esxcfg-vswitch -L vmnic4 vSwitch1

/usr/sbin/esxcfg-vswitch -U vmnic3 vSwitch2

/usr/sbin/esxcfg-vswitch -U vmnic5 vSwitch2

/usr/sbin/esxcfg-vswitch -L vmnic3 vSwitch2

/usr/sbin/esxcfg-vswitch -L vmnic5 vSwitch2

/usr/bin/vimsh -n -e /internalsvc/refresh_network

[jasonboche]

Flagged thread for VMware to look at.

Jason Boche

VMware Communities User Moderator

[mpverr]

I was playing around with vimsh as well and found the same problem as it relates to the fact that esxcfg-vswitch -l no longer provides you with the internal port group. However, after you create you vmkernel port (esxcfg-vmknic -a -i ........ VMotion) then if you do a esxcfg-vmknic -l you will see there is a new column cited:

Interface Port Group IP Address Netmask Broadcast MAC Address MTU TSO MSS Enabled

vmk0 VMotion 1500 40960 true

the interface, in this case is vmk0. So you should be able to extract that information doing something like:

vimsh -n -e "/hostsvc/vmotion/vnic_set `esxcfg-vmknic -l | awk '/vmk/ {print $1}'`"

after running this it shows up in VIC that VMotion is now enabled.

[mpverr]

I'm finding this does not work.

after you unlink vmnic1, then relink it, it dhows vmnic1 in standby mode.

then after you do the service restart and wait a minute or two, i try the next commands.

vimsh -n -e "hostsvc/net/vswitch_setpolicy --nicorderpolicy-active vmnic0 vSwitch0"

vimsh -n -e "hostsvc/net/vswitch_setpolicy --nicorderpolicy-active vmnic0,vmnic1 vSwitch0"

vmnic1 still stays in stand by

[Da_Monkey80]

I noticed that there is a new command option for esxcfg-vswitch -M in 3.5

After messing around with my script I notice that if you use the "-M" first before the "-L" that seems to add all the adapters as "Active"

example:

esxcfg-vswitch -M vmnic1 vSwitch1 -pNetwork0

esxcfg-vswitch -M vmnic3 vSwitch1 -pNetwork0

esxcfg-vswitch -L vmnic1 vSwitch1

esxcfg-vswitch -L vmnic3 vSwitch1

(-p being the port group name)

Just thought you all might want to know

[admin]

The issue is actually a policy setting called maxActive. This value went from optional in 3.0 to required in 3.5 If you had the value set in 3.0 then you're network was unchanged, but if you didn't have it set in 3.0 we had to give you a default value.

[admin]

esxcfg-vswitch doesn't show an internal portgroup name anymore because it doesn't exist anymore.

[mpverr]

As with most things there are multiple ways to complete a task. I guess it's whatever you would prefer. I have found that sometimes in doing the Link/Unlink that the last nic still stays in standby mode. So if your process works for you then I'd use it if you're more comfortable. At the very least, the entry I cited for enabling VMotion works perfectly. Ho Ho Ho - Merry Christmas everyone.

[mpverr]

I know a lot of you are "guru's" and don't need the assistance, I have sent this script out to others in the past to those less experienced and they have said it was very helpful. So for anyone looking to automate the creation/configuration of the networking with ESX (in this case ESX 3.5) please see that attached snippet of my shell script.

With my systems we have 6 nics. Two are bonded for the console/VMotion and the other four are bonded for Virtual machines. On the cisco switch I have the two bonds configured as etherchannel port groups. In the script are references to Changing the console memory, creating accounts, configuring DNS and assigning VMotion IP information. If there is any interest in seeing those let me know and I'll be happy to upload those as well. I'm not at least right now as it would require effort on my part to strip out system names and IP info. Enjoy.

[esiebert7625]

Guru's don't know everything, I'm constantly learning new things about ESX. That's one of the great benefits of these forums, learning from other users knowledge and experiences.

Eric Siebert

VMware Communities User Moderator

-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-=-

Visit my website:

-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-=-

[mpverr]

VIMSH 3.5 from Xtravirt was just released!

http://www.xtravirt.com/index.php?option=com_remository&Itemid=75&func=download&id=21&chk=4738b7ad74...

[sbeaver]

Do you have all the port groups created before you unlink and relink?

[mpverr]

I'm quoting from the latest Xtravit 3.5 vimsh document on page 21 ....

"An issue which can arise is that normally to place a vmnic into the 'unused adapters' set you simply exclude the particular adapter from the --nicorderpolicy command. However, when there is only one adapter assigned there is no obvious method to remvoing it as the command requires an argument.

The other problem is that when you use the esxcfg-vswitch to unlink the vmnic from the vswitch it doesn't update the vswitch state info say that more than 1 vmnic can be active, therefore when you re-add the vminc back in it goes striaght to standby."

The rest of that page provides solutions to this. While this may not happen in your specific environment, it has been confirmed (by myself and others) that it does in fact happen. As with anything, use the solution the works for you. We have taken the approach suggested in the Xtravirt document just to ensure we don't have any anomolies.

[srbush]

I had been struggling with the issue of the 2 nics on my SAN vswitch not being active after a scripted install.

I tried many of the work arounds mentioned but the ones using the following did not work for me:

- adding the "-M" option to the esxcfg-vswitch command

- using vimsh - in fact none of the vimsh commands appeared to work properly, I always got a "file not found" error before the status listing

I did get the unlink/relink method discussed in this thread to work consistantly........

Thanks to all for the discussions....