VMware Cloud Community
rkarel3
Contributor
Contributor
‎07-01-2008 10:23 AM

esxcfg-vswitch to re-link vmnic's to SC

I can not re-link using the "L" option after un-linking them using the U option.

All to test secondary SC wich is not available after un-linking vmnic's

0 Kudos
15 Replies
kjb007
Immortal
Immortal
‎07-01-2008 10:33 AM

What error are you getting?

Run esxcfg-vswitch -l to make sure you're not seeing it. Also, run esxcfg-nics -l to make sure ESX still sees the pNIC.

-KjB

vExpert/VCP/VCAP vmwise.com / @vmwise -KjB
0 Kudos
gary1012
Expert
Expert
‎07-01-2008 10:36 AM

esxcfg-vswitch -L vmnicX vSwitchX is the syntax to reassign a NIC to the SC. Are you at the KVM console when enter in the syntax. Also, did the NIC removal delete the IP from the vswif?

Community Supported, Community Rewarded - Please consider marking questions answered and awarding points to the correct post. It helps us all.
0 Kudos
rkarel3
Contributor
Contributor
‎07-01-2008 11:09 AM

Used the esxcfg-vswitch -U vmnic0 vSwitch0 to unlink nic to test HA.

I can not get it back using the L option.

0 Kudos
rkarel3
Contributor
Contributor
‎07-01-2008 11:12 AM

When I ran esxcfg-vswitch -U vmnic0 vSwitch0 When I tried to re-link it, it would not work.

0 Kudos
kjb007
Immortal
Immortal
‎07-01-2008 11:13 AM

Please post the output of esxcfg-vswitch -l and esxcfg-nics -l

-KjB

vExpert/VCP/VCAP vmwise.com / @vmwise -KjB
0 Kudos
sbeaver
Leadership
Leadership
‎07-01-2008 11:16 AM

Here is part of one of my scripts that will tear down networking an rebuild so hopefully something out of this can help point you in the right direction

setConsoleNIC()

{

echo "Deleting current network configuration" >> /root/PostInstall/PostInstall.log

/usr/sbin/esxcfg-vswitch vSwitch0 -D "VM Network"

/usr/sbin/esxcfg-vswitch -U vmnic0 vSwitch0

/usr/sbin/esxcfg-vswitch -U vmnic1 vSwitch0

/usr/sbin/esxcfg-vswif -d vswif0

/usr/sbin/esxcfg-vswitch -d vSwitch0

vmware-vim-cmd "hostsvc/net/refresh"

vmware-vim-cmd "internalsvc/refresh_network"

echo "Reconfiguring Service Console NIC..." >> /root/PostInstall/PostInstall.log

cp /etc/vmware/esx.conf /tmp/post/esx.conf.bak

/usr/sbin/esxcfg-vswitch -a vSwitch0

vmware-vim-cmd "hostsvc/net/vswitch_setpolicy --nicorderpolicy-active vmnic0,vmnic1 vSwitch0"

vmware-vim-cmd "hostsvc/net/refresh"

vmware-vim-cmd "internalsvc/refresh_network"

/usr/sbin/esxcfg-vswitch vSwitch0 -L vmnic0

/usr/sbin/esxcfg-vswitch vSwitch0 -L vmnic1

/usr/sbin/esxcfg-vswitch -U vmnic0 vSwitch0

/usr/sbin/esxcfg-vswitch -U vmnic1 vSwitch0

/usr/sbin/esxcfg-vswitch vSwitch0 -L vmnic0

/usr/sbin/esxcfg-vswitch vSwitch0 -L vmnic1

vmware-vim-cmd "hostsvc/net/refresh"

vmware-vim-cmd "internalsvc/refresh_network"

/usr/sbin/esxcfg-vswitch -A "Service Console" vSwitch0

/usr/sbin/esxcfg-vswitch vSwitch0 -p "Service Console" -v 0

/usr/sbin/esxcfg-vswif -a vswif0 -p "Service Console" -i $SCIP -n 255.255.255.0

route add default gw $DGW

vmware-vim-cmd "hostsvc/net/refresh"

vmware-vim-cmd "internalsvc/refresh_network"

}

Steve Beaver

VMware Communities User Moderator

====

Co-Author of "VMware ESX Essentials in the Virtual Data Center"

Coming soon to a store near you!

*Virtualization is a journey, not a project.*

Steve Beaver
VMware Communities User Moderator
VMware vExpert 2009 - 2020
VMware NSX vExpert - 2019 - 2020
====
Co-Author of "VMware ESX Essentials in the Virtual Data Center"
(ISBN:1420070274) from Auerbach
Come check out my blog: [www.virtualizationpractice.com/blog|http://www.virtualizationpractice.com/blog/]
Come follow me on twitter http://www.twitter.com/sbeaver

**The Cloud is a journey, not a project.**
rkarel3
Contributor
Contributor
‎07-01-2008 12:02 PM

Thank you very much it is back. How would you test the secondary SC? When I downed the vswif0, I could no longer ping vswif1 SC.

0 Kudos
rkarel3
Contributor
Contributor
‎07-01-2008 12:14 PM

# ./esxcfg-vswitch -l

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks

vSwitch0 64 5 64 1500 vmnic2,vmnic0

PortGroup Name VLAN ID Used Ports Uplinks

Service Console 0 1 vmnic2,vmnic0

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks

vSwitch1 64 6 64 1500 vmnic3,vmnic1

PortGroup Name VLAN ID Used Ports Uplinks

Service Console 2 2 1 vmnic3,vmnic1

vmotion 2 1 vmnic1,vmnic3

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks

vSwitch2 128 10 128 1500 vmnic11,vmnic10,vmnic9,vmnic8,vmnic7,vmnic6,vmnic5,vmnic4

PortGroup Name VLAN ID Used Ports Uplinks

vm_75 10 0 vmnic4,vmnic5,vmnic6,vmnic7,vmnic8,vmnic9,vmnic10,vmnic11

vm_74 9 0 vmnic4,vmnic5,vmnic6,vmnic7,vmnic8,vmnic9,vmnic10,vmnic11

vm_73 8 0 vmnic4,vmnic5,vmnic6,vmnic7,vmnic8,vmnic9,vmnic10,vmnic11

vm_72 7 0 vmnic4,vmnic5,vmnic6,vmnic7,vmnic8,vmnic9,vmnic10,vmnic11

vm_71 6 0 vmnic4,vmnic5,vmnic6,vmnic7,vmnic8,vmnic9,vmnic10,vmnic11

vm_70 5 0 vmnic4,vmnic5,vmnic6,vmnic7,vmnic8,vmnic9,vmnic10,vmnic11

vm_69 4 0 vmnic4,vmnic5,vmnic6,vmnic7,vmnic8,vmnic9,vmnic10,vmnic11

vm_68 3 0 vmnic4,vmnic5,vmnic6,vmnic7,vmnic8,vmnic9,vmnic10,vmnic11

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks

vSwitch3 64 1 64 1500

PortGroup Name VLAN ID Used Ports Uplinks

Internal 0 0

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks

vSwitch4 64 1 64 1500

PortGroup Name VLAN ID Used Ports Uplinks

Isolated Network 0 0

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks

vmnic2 64 1 64 1500

PortGroup Name VLAN ID Used Ports Uplinks

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks

vmnic0 64 1 64 1500

PortGroup Name VLAN ID Used Ports Uplinks

# esxcfg-vswitch -l

bash: esxcfg-vswitch: command not found

# ./esxcfg-vswitch -l

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks

vSwitch0 64 5 64 1500 vmnic2,vmnic0

PortGroup Name VLAN ID Used Ports Uplinks

Service Console 0 1 vmnic2,vmnic0

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks

vSwitch1 64 6 64 1500 vmnic3,vmnic1

PortGroup Name VLAN ID Used Ports Uplinks

Service Console 2 2 1 vmnic3,vmnic1

vmotion 2 1 vmnic1,vmnic3

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks

vSwitch2 128 10 128 1500 vmnic11,vmnic10,vmnic9,vmnic8,vmnic7,vmnic6,vmnic5,vmnic4

PortGroup Name VLAN ID Used Ports Uplinks

vm_75 10 0 vmnic4,vmnic5,vmnic6,vmnic7,vmnic8,vmnic9,vmnic10,vmnic11

vm_74 9 0 vmnic4,vmnic5,vmnic6,vmnic7,vmnic8,vmnic9,vmnic10,vmnic11

vm_73 8 0 vmnic4,vmnic5,vmnic6,vmnic7,vmnic8,vmnic9,vmnic10,vmnic11

vm_72 7 0 vmnic4,vmnic5,vmnic6,vmnic7,vmnic8,vmnic9,vmnic10,vmnic11

vm_71 6 0 vmnic4,vmnic5,vmnic6,vmnic7,vmnic8,vmnic9,vmnic10,vmnic11

vm_70 5 0 vmnic4,vmnic5,vmnic6,vmnic7,vmnic8,vmnic9,vmnic10,vmnic11

vm_69 4 0 vmnic4,vmnic5,vmnic6,vmnic7,vmnic8,vmnic9,vmnic10,vmnic11

vm_68 3 0 vmnic4,vmnic5,vmnic6,vmnic7,vmnic8,vmnic9,vmnic10,vmnic11

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks

vSwitch3 64 1 64 1500

PortGroup Name VLAN ID Used Ports Uplinks

Internal 0 0

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks

vSwitch4 64 1 64 1500

PortGroup Name VLAN ID Used Ports Uplinks

Isolated Network 0 0

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks

vmnic2 64 1 64 1500

PortGroup Name VLAN ID Used Ports Uplinks

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks

vmnic0 64 1 64 1500

PortGroup Name VLAN ID Used Ports Uplinks

# ./esxcfg-nics -l

Name PCI Driver Link Speed Duplex MTU Description

vmnic0 0a:04.00 tg3 Up 1000Mbps Full 1500 Broadcom Corporation HP NC326i Integrated Dual Port PCI Express Gigabit Server Adapter

vmnic2 05:00.00 bnx2 Up 1000Mbps Full 1500 Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-SX

vmnic1 03:00.00 bnx2 Up 1000Mbps Full 1500 Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-SX

vmnic3 0a:04.01 tg3 Up 1000Mbps Full 1500 Broadcom Corporation HP NC326i Integrated Dual Port PCI Express Gigabit Server Adapter

vmnic4 14:04.00 tg3 Up 1000Mbps Full 1500 Broadcom Corporation Broadcom BCM5715S Gigabit Ethernet

vmnic5 14:04.01 tg3 Up 1000Mbps Full 1500 Broadcom Corporation Broadcom BCM5715S Gigabit Ethernet

vmnic6 16:04.00 tg3 Up 1000Mbps Full 1500 Broadcom Corporation Broadcom BCM5715S Gigabit Ethernet

vmnic7 16:04.01 tg3 Up 1000Mbps Full 1500 Broadcom Corporation Broadcom BCM5715S Gigabit Ethernet

vmnic8 1b:04.00 tg3 Up 1000Mbps Full 1500 Broadcom Corporation Broadcom BCM5715S Gigabit Ethernet

vmnic9 1b:04.01 tg3 Up 1000Mbps Full 1500 Broadcom Corporation Broadcom BCM5715S Gigabit Ethernet

vmnic10 1d:04.00 tg3 Up 1000Mbps Full 1500 Broadcom Corporation Broadcom BCM5715S Gigabit Ethernet

vmnic11 1d:04.01 tg3 Up 1000Mbps Full 1500 Broadcom Corporation Broadcom BCM5715S Gigabit Ethernet

#

0 Kudos
kjb007
Immortal
Immortal
‎07-01-2008 12:22 PM

Just an observation here, but I think you're asking for a future migraine having that many pNICs in a team. Hopefully, you have spread the Active pNICs over the available VLANs, but even still, troubleshooting in that environment will be a NIGHTMARE!

Also, in the case of performance, you will be hard pressed to see much of a performance improvement spreading I/O over 8 pNICs as opposed to 2. Instead of what you have, I would pick a couple of VLANs, and assign them a pair of pNICs, and then spread the VLANs over portgroup that have 2 pNICs behind them. Believe me, the first time you run into a networking issue, you will thank me.

Also, as I stated previously, you don't need to have 2 service console ports. If you have a pair of pNICs, then the redundancy is accomplished in hardware, and not in the software. Also, to test redundancy, I would not recommend disabling the pNIC from the team, instead, I would disable the port on the switch side, or simply pull a cable out of the interface.

-KjB

vExpert/VCP/VCAP vmwise.com / @vmwise -KjB
0 Kudos
dconvery
Champion
Champion
‎07-02-2008 05:33 AM

Rich -

So, the syntax was " /usr/sbin/esxcfg-vswitch vSwitch0 -L vmnic0" ???

Dave

Dave Convery, VCDX-DCV #20 ** http://www.tech-tap.com ** http://twitter.com/dconvery ** "Careful. We don't want to learn from this." -Bill Watterson, "Calvin and Hobbes"
0 Kudos
rkarel3
Contributor
Contributor
‎07-03-2008 04:49 AM

The -L option would work given it was an unused vmnic.

The best option for me was to delete the virtaul switcha and recreate it. However, before I could do it, the host mysteriously made a connection to VC through the secondary service console. I was then able to use the VC client to re-assign the NICs.

It took about an hour to re-establish the connection through the secondary SC. I could not ping the secondary SC after un-linking primary from its NICs.

There is no DNS entry for the secondary SC.

0 Kudos
dconvery
Champion
Champion
‎07-03-2008 04:53 AM

Sux...Another reason why a NIC team is better for the SC than two separate SC configs....

Dave Convery, VCDX-DCV #20 ** http://www.tech-tap.com ** http://twitter.com/dconvery ** "Careful. We don't want to learn from this." -Bill Watterson, "Calvin and Hobbes"
0 Kudos
rkarel3
Contributor
Contributor
‎07-03-2008 04:53 AM

The second SC was for additional connectivity to protect against an pre-mature HA isolation. There are 12 NICs in these blades. I believe that 4 NICs are sufficient for the production team and the client finally agreed. The answer for me was to stop networking on the host completely to isolate the host and test HA failover without causing damage to the host configuration or a hard power down.

THANKS!!!!!!

0 Kudos
rkarel3
Contributor
Contributor
‎07-03-2008 04:58 AM

It was Mike's suggestion and I am in no position to go against the suggestion of a practice director. I agree with you however.

0 Kudos
dconvery
Champion
Champion
‎07-03-2008 05:05 AM

gotcha..

Dave Convery, VCDX-DCV #20 ** http://www.tech-tap.com ** http://twitter.com/dconvery ** "Careful. We don't want to learn from this." -Bill Watterson, "Calvin and Hobbes"
0 Kudos