HP 5308XL, LACP & ESX 3.5

[Marv_Gordon]

I've followed the instructions and have set three static LACP trunked ports on our Procurve 5308XL (very latest firmware).

I set route based on IP hash for the Vswitch as documented.

Everything seems to work fine until I reboot my test ESX server. Can't ping anything after that.

Anyone else running a similar config?

Note: I do have 2 ports on one gigabit module in the 5308xl and 1 in another gigabit module on the same switch. Might this be the issue ?

[mike_laspina]

Hi,

#trunk B1,F15 Trk1 LACP

This command is actually not correct for a static trunk. The LACP key word specifies use LACP protocol to create a trunk.

Change it to the following.

#trunk B1,F15 Trk1 trunk

[champcf]

I'm using a ProCurve 2824 switch and I'm having similar problems. When using "trunk", I sometimes can ping and sometimes can't. When I unplug 1 cable so that there's only 1 cable from ESX server to the switch, I can ping perfectly fine. After plugging in the other cable, it works fine until may 15-45 minutes later. Then, pings no longer respond. If I unplug, pings come back. Plug it back in, pings are flowing for another 15-45 minutes. After changing to "FEC" on the switch, everything works perfectly. Of course, FEC will be gone after upgrading the firmware.

Does this sound like there is some kind of broadcast storm or loop going on? Will setting up a spanning tree thingy work?

[JHAPrins]

I was just searching around with the same problem, also with a 2824.

If there is anyone here that can give a proper sollution how to connect an ESX server to a HP Procurve, I would be very happy.

I allready have the latest firmware so I can't configure FEC anymore.

Jan Hugo Prins

[mike_laspina]

Can you expand on what your configuration looks like and how you want your esx server to use the trunk?

The example

trunk B1,F15 Trk1 trunk

Is for HP switches

here ports B1 and F15 are to be members of trk1 and the mode is static using the key word trunk on the end of the command

[JHAPrins]

It's just a very basic trunk:

trunk 11-12 Trk8 Trunk

interface 11

no lacp

exit

interface 12

no lacp

exit

# esxcfg-vswitch -l

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks

vSwitch1 64 8 64 1500 vmnic0,vmnic1

PortGroup Name VLAN ID Used Ports Uplinks

StorageNet 16 1 vmnic1,vmnic0

Service Console 200 1 vmnic1,vmnic0

BB-LAN 0 1 vmnic1,vmnic0

VMkernel 16 1 vmnic1,vmnic0

# esxcfg-nics -l

Name PCI Driver Link Speed Duplex MTU Description

vmnic1 06:00.01 e1000 Down 0Mbps Half 1500 Intel Corporation 80003ES2LAN Gigabit Ethernet Controller

vmnic0 06:00.00 e1000 Up 1000Mbps Full 1500 Intel Corporation 80003ES2LAN Gigabit Ethernet Controller

The problem is that when I connect it this way, with just the ESX server with 2 connects to this switch I lose ping to the server and the VM's after a while.

What I would like is that both interfaces are used so I have 2Gbit loadbalanced.

Greetings,

Jan Hugo Prins

[mike_laspina]

Trunking in this manner will not give you 2G over one VM network path. You are probably better off setting up a Team on the ESX host and activate them both. The hypervisor will try to balance the load across the two nics by alternating the VM assignments on VM initialization.

I one system I had to reboot it before it work start working.

On HP switches I have noticed that the VLAN tagging assignments get toasted when you change the trunk assignment

e.g.

port 1 & 2 are tagged for VLAN 100

trunk 1-2 trk1 trunk

You will now need to re-tag trk1 for VLAN 100

[champcf]

The trunk setting is mysteriously working today and I haven't gotten cut off yet. Well, I checked the port counters to see what was actually going on. It turns out that one port is doing all the receiving and the other port is doing all the transmitting. That isn't very satisfying because this is only as good as half-duplex. I don't have enough hardware at the moment to test if both ports will act as full duplex if demanded.

[JHAPrins]

Could you please explain me how to setup a team on the ESX host?

Because if this is what it sounds like it is then it is exactly what I want.

I just found the following:

"Creating the NIC team on the ESX Server side is as simple as adding

physical NICs to the vSwitch and setting the load balancing policy

appropriately. At this point, the NIC team should be working."

Is this correct?

And if I do this, is then the trunk setting from previous messages also the way to go?

Greetings,

Jan Hugo Prins

[w0d0l0ner]

Hi Mike,

I just reread this whole thread and I think might mislead some people.

The command "trunk B1,F15 Trk1 lacp" specifies exactly what I really would like.

Well, actually what I wanted to use was "trunk A1,A3,B1,B3,F15 Trk1 lacp". (5 ports)

The HP switches I have, 5308/4XL, can only bundle 4 ports at a time in one trunk.

I don't know if this applies to other switch models or not.

However LACP can have extra standby links so that if a link dies, you have a replacement ready and waiting.

The ESX can't however speak LACP.

Sometimes LACP is refered to as 802.3ad, however LACP is only one part of the 802.3ad specification.

After reading some parts of HP documentation, the distinction is not at all clear.

Somewhere in the ESX documentation I read it said that you need an 802.3ad compatible switch.

This lead me then to understand it supported the LACP.

Since reading your post, I changed the setup of the trunks to be of the "trunk" type, and show running doesn't show those ports as having the dynamic lacp activated.

I plan on bringing our ESX's down soon for maintenance (20080719).

I can't wait to see if everything works afterward or not.

I have however already had this configuration, and I had connectivity problems after a reboot.

As I described, not all IP's on the box can be reached without problems from all IP's off the box.

(some hosts would get arp replies from the ESX and others would not)

Just curios, what ROM and software version are you running?

I'm on E.05.05 / E.10.67, but I could update the switches to a newer version.

[champcf]

You should be able to find out how to set up a NIC team easily with Google, but here's my version. Go into the configuration tab of the host. Click the Networking link. Locate whichever vSwitch you want to use and click Properties. Edit the the vSwitch. Go to the NIC Teaming tab. For Load Balancing, choose Route based on ip hash. This is also the place where you would add NICs to the vSwitch. If you don't have a vSwitch for your cards yet, you'll want to Add Networking and you can choose multiple NICs from there.

[mike_laspina]

I have nic teams setup on static trucks with no real issues and also with 802.1q only (route on Port ID only with 802.1q).

Trunks allow the ability to route based on ip which is some what better than Port ID because its more dynamic. Port ID will stay on a port for the duration of the VM's network binding.

For trunks you must use fast STP if you have STP enabled or the ports will not always come up on reboot.

This is working on HP models 5308, 2900, 2800 all over a variety of firmware versions. Q.10.01, T.13.09, E.08.42, N.11.09 etc.

To setup up the team go into the ESX Configuration panel in VC, select networking-> properties of the vswitch-> dclick on the vSwitch item and click on the nic teaming tab.