VMware Cloud Community
SebDooris
Contributor
Contributor

ESXi 5.0 Nic teaming on a Cisco 3750

Hi

Let me just start by saying I'm not a networks guy so be gentle.

I'm trying to configure a Cisco 3750G Version 12.2(44)SE5 with my new ESXi server.

When setting up "Configure Managment Network" I select vmnic0 and vmnic1 and dont set the VLAN but I get no traffic.

Have verified that that vlan160 traffic is carried on the switch and have tried setting the vlan on the esx with no luck.

This is my current config from the 3750.

================================================================================

#show etherchannel summary

3      Po3(SU)          -        Gi1/0/5(P)  Gi1/0/6(P)

================================================================================

#show etherchannel protocol

Group: 3
----------
Protocol:   -  (Mode ON)

================================================================================

#show etherchannel 3 detail
Group state = L2
Ports: 2   Maxports = 8
Port-channels: 1 Max Port-channels = 1
Protocol:    -
Minimum Links: 0
                Ports in the group:
                -------------------
Port: Gi1/0/5
------------

Port state    = Up Mstr In-Bndl
Channel group = 3           Mode = On              Gcchange = -
Port-channel  = Po3         GC   =   -             Pseudo port-channel = Po3
Port index    = 0           Load = 0x00            Protocol =    -

Age of the port in the current state: 0d:19h:19m:36s

Port: Gi1/0/6
------------

Port state    = Up Mstr In-Bndl
Channel group = 3           Mode = On              Gcchange = -
Port-channel  = Po3         GC   =   -             Pseudo port-channel = Po3
Port index    = 0           Load = 0x00            Protocol =    -

Age of the port in the current state: 0d:19h:19m:36s

                Port-channels in the group:
                ---------------------------

Port-channel: Po3
------------

Age of the Port-channel   = 0d:19h:19m:56s
Logical slot/port   = 10/3          Number of ports = 2
GC                  = 0x00000000      HotStandBy port = null
Port state          = Port-channel Ag-Inuse
Protocol            =    -
Port security       = Disabled

Ports in the Port-channel:

Index   Load   Port     EC state        No of bits
------+------+------+------------------+-----------
  0     00     Gi1/0/5  On                 0
  0     00     Gi1/0/6  On                 0

Time since last port bundled:    0d:19h:19m:36s    Gi1/0/6
Time since last port Un-bundled: 0d:19h:19m:56s    Gi1/0/6

================================================================================

#show running-config
port-channel load-balance dst-ip
spanning-tree mode pvst
spanning-tree extend system-id
!

interface GigabitEthernet1/0/5
switchport access vlan 160
switchport mode access
channel-group 3 mode on
!
interface GigabitEthernet1/0/6
switchport access vlan 160
switchport mode access
channel-group 3 mode on

!

Reply
0 Kudos
11 Replies
Rubeck
Virtuoso
Virtuoso

At a quick glance the config looks fine, but have you set the ESXi load balancing to Route based on IP hash on the vSwitch side?

You do not mention this, although I would think you already have..

/Rubeck

Reply
0 Kudos
SebDooris
Contributor
Contributor

Hi Rubeck

Thanks for the reply I have not seen the option for "Route based on IP hash" on the text concole screen and am unable to find any mention of it. As this server has not yet been assigned a managment IP due to this problem I do not have access to the web/VC interface.

Do you know of a way to alter vSwitch modes from the console?

Seb.

Reply
0 Kudos
Baddos
Enthusiast
Enthusiast

Switch config looks ok, but what you need to do next is either disable of unplug one of the switchports. Now you should be able to ping/management the host and setup the network config. In the configuration, set the  vswitch nic teaming configuration  to load balance based upon IP hash. Click on the management network nic teaming, and make sure the load balancing for that is unchecked so it inherits it's setting from the vswitch.

Reply
0 Kudos
Rubeck
Virtuoso
Virtuoso

Ohh... I see.... As Im still trying to learn the ESXi 5 CLI command set, I cant really say for sure.... But configuring route based on ip hash in ESXi 4 is something like "vim-cmd hostsvc/net/vswitch_setpolicy --nicteaming-policy=loadbalance_ip vSwitchX" . Best I've got for now as I dont have access to any ESXi5 hosts at the moment..

Ohh BTW.. Your physical switch ports need some additional config too, although it has nothing to do with your issue...  You need to exclude these from spanning tree ops by adding "spanning tree portfast" to those ports...  And also you need "switchport nonegotiate" on them too, as ESXi dosn't support dynamic trunking.. (not that this is required, but might save you from trouble later on)

All I've got at the moment.. Sorry..

/Rubeck

Rubeck
Virtuoso
Virtuoso

A how to is here: http://kb.vmware.com/kb/1011520

/Rubeck

SebDooris
Contributor
Contributor

Thank you everyone for all your help I've made quite a few chnages as recommended but am still having problems. The vSwitch is setup for route based ip hash but when this config is actavated all traffic on both the interfaces cease.

Any suggestions?

#show etherchannel summary

=======================================================================================

3      Po3(SD)          -        Gi1/0/5(s)  Gi1/0/6(s)

=======================================================================================

#show etherchannel protocol

Group: 3
----------
Protocol:   -  (Mode ON)

=======================================================================================

#show etherchannel 3 detail
Group state = L2
Ports: 2   Maxports = 8
Port-channels: 1 Max Port-channels = 1
Protocol:    -
Minimum Links: 0
                Ports in the group:
                -------------------
Port: Gi1/0/5
------------

Port state    = Up Cnt-bndl Suspend Not-in-Bndl
Channel group = 3           Mode = On              Gcchange = -
Port-channel  = null        GC   =   -             Pseudo port-channel = Po3
Port index    = 0           Load = 0x00            Protocol =    -

Age of the port in the current state: 0d:17h:47m:31s

Probable reason: dtp nonegotiate of Gi1/0/5 is 1, Po3 id 0
Port: Gi1/0/6
------------

Port state    = Up Cnt-bndl Suspend Not-in-Bndl
Channel group = 3           Mode = On              Gcchange = -
Port-channel  = null        GC   =   -             Pseudo port-channel = Po3
Port index    = 0           Load = 0x00            Protocol =    -

Age of the port in the current state: 0d:17h:47m:31s

Probable reason: dtp nonegotiate of Gi1/0/6 is 1, Po3 id 0
                Port-channels in the group:
                ---------------------------

Port-channel: Po3
------------

Age of the Port-channel   = 0d:17h:49m:36s
Logical slot/port   = 10/3          Number of ports = 0
GC                  = 0x00000000      HotStandBy port = null
Port state          = Port-channel Ag-Not-Inuse
Protocol            =    -
Port security       = Disabled

Time since last port bundled:    4d:17h:41m:44s    Gi1/0/6
Time since last port Un-bundled: 0d:18h:16m:27s    Gi1/0/6

=======================================================================================

#show running-config

!
port-channel load-balance src-dst-ip
spanning-tree mode pvst
spanning-tree extend system-id
!

interface Port-channel3
description ESXGroup
switchport access vlan 160
switchport mode access
!

interface GigabitEthernet1/0/5
description vmnic0
switchport access vlan 160
switchport mode access
switchport nonegotiate
channel-group 3 mode on
spanning-tree portfast
!
interface GigabitEthernet1/0/6
description vmnic1
switchport access vlan 160
switchport mode access
switchport nonegotiate
channel-group 3 mode on
spanning-tree portfast
!

Reply
0 Kudos
Rubeck
Virtuoso
Virtuoso

Can you verify that portgroups within the vSwitch are also configured with route based on IP hash...?

/Rubeck

Reply
0 Kudos
SebDooris
Contributor
Contributor

Yep all checks out

vSphere standard switch

vSwitch0 = Route based on IP hash

Management Network = Route based on IP hash

Reply
0 Kudos
Rubeck
Virtuoso
Virtuoso

Hmmm... thinking thinking... Cant really figure out issue might be..

Ill be back if I find some thing..

/Rubeck

Reply
0 Kudos
nirvy
Commander
Commander

#show etherchannel summary

=======================================================================================

3      Po3(SD)          -        Gi1/0/5(s)  Gi1/0/6(s)

=======================================================================================


This says your port channel is down at layer2 and both ports are currently suspended.  Can you do a show run interface port-channel3 and show us the output.

Did you enable port-channeling on your hosts using the cli or via the gui?  Maybe you need to restart/refresh the networking for the vim-cmd to take...

some old working configs I have are as follows:

port-channel load-balance src-dst-ip

interface Port-channel1
switchport trunk encapsulation dot1q
switchport mode trunk
!

interface GigabitEthernet1/0/1
description Trunk link to Core ESXi host 1
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 1 mode on
!
interface GigabitEthernet2/0/1
description Trunk link to Core ESXi host 1
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 1 mode on
!

What you want to see:

Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
1      Po1(SU)          -        Gi1/0/1(P)  Gi2/0/1(P)

Port-channel1 is up, line protocol is up (connected)
  Hardware is EtherChannel, address is f025.7204.8581 (bia f025.7204.8581)
  MTU 1500 bytes, BW 2000000 Kbit, DLY 10 usec,

Reply
0 Kudos
SebDooris
Contributor
Contributor

Hi Nirvy

Thanks for the reply.

#show run interface port-channel3
Building configuration...

Current configuration : 111 bytes
!
interface Port-channel3
description ESXGroup
switchport access vlan 160
switchport mode access
end

Yea the ports go into suspended mode once I enable the config on the esx. I configured the esx via the gui and cli but may try via "Tech Support Mode" next.

Thanks

Reply
0 Kudos