Kindred_VMSuppo
Contributor
Contributor

HP Blade Servers utilizing Virtual Connect

Jump to solution

Is anyone running the HP Blade servers and utilizing Virtual Connect yet? We are looking at getting away from the DL580's and move to blades but we do not want to be the first ones to bring it up with Virtual Connect.

We are looking for any gotcha's, praise or ease of use or lack there of.

Any help would be much appreciated.

0 Kudos
1 Solution

Accepted Solutions
Jae_Ellers
Virtuoso
Virtuoso

We've had C-Class running 460c & 465c blades for several months now. We started out of the box with Virtual Connect for networking and it's worked great.

I have 2 vc modules in each chassis. We have 4 uplink ports aggregated on each vc module. We have an additional trunked link on one module for a special purpose ESX network link (test network and vmotion on a single line).

I did put in the feature request as another poster specified. Currently if you create shared uplinks that are trunked there is no option to pass the tags thru for some of the networks. It's an either or situation. You can have your cake or you can eat it, but you can't have some networks with multiple tags and some without. You get to split the vlans off onto virtual networks, but can't pass them all thru to ESX. I think HP has heard about this and I hope this feature is forthcoming.

You can pass all vlans thru, but then you have to make sure the non ESX servers can peel the right vlan traffic off and you are dependent on the HP network software and admin configuration to do the right thing. Currently it's easier to separate trunked and non-trunked onto separate uplinks.

Other than this pretty obvious oversight in the base functionality it's easy to use and we have not had a hitch in the networking so far.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=- http://blog.mr-vm.com http://www.vmprofessional.com -=-=-=-=-=-=-=-=-=-=-=-=-=-=-

View solution in original post

0 Kudos
17 Replies
spex
Expert
Expert

One problem - if you mix different blades (HP and IBM):

Each i/o port in your blade is connected to an individual i/o bay in the backend. Means if you have 6 nics you need 6 virtual connect modules.

Worst case: You have one esx with 6 nic's and the rest of your blades use only on nic Most ports of your virtual connect modules you bought are unused.

Despite that I think very powerful technology.

Regards Spex

vmrulz
Hot Shot
Hot Shot

I would have to disagree regarding the port mapping spex. Virtual Connect Manager allows you to create shared uplink sets that are useable by any blade in the enclosure. I'm sandboxing a c Class right now working through the details of virtual connect.. very cool tech for sure.

The part I'm trying to figure out is how to setup the uplink set to NOT trim VLAN tags from the shared uplink and just pass them to ESX for use in port groups.

Take note that from the SAN Virtual Connect side of things your fabric switches must be able to talk NPIV which is a relatively new port aggregation "protocol" allowing multiple WWD's on a single port. We run Cisco MDS switches which don't yet have NPIV but we're soon to upgrade to SAN-OS v3.x which has NPIV.

Kindred_VMSuppo
Contributor
Contributor

vmrulz. What hp servers are you using? 460c?

Do you have esx servers running under virtual connect mgr or are you still working out the configs?

Any gotchas that I should look out for?

0 Kudos
vmrulz
Hot Shot
Hot Shot

We started by demo'ing a c Class from HP through our VAR then ended up buying it. If you can pull it off, I'd try doing it that way. You'll end up wanting to buy it when you're done but at least you have the option of sending it all back.

We're an AMD shop and only have the 465 and 685 blades. We are still very early in the sand boxing stage working out procedures and getting people trained on the hardware.

Other gotchas might include deciding on whether you want VC to create virtual MAC and WWD's or use burned in ID's.. either way will work, its just different from any other hardware device I've used.. very similar in design to how ESX works from a hardware virtualization standpoint.

Keep in mind this is still bleeding edge technology but from what I can see VC is a winner.. in the end simplifying server provisioning for us server guys.

0 Kudos
Jae_Ellers
Virtuoso
Virtuoso

We've had C-Class running 460c & 465c blades for several months now. We started out of the box with Virtual Connect for networking and it's worked great.

I have 2 vc modules in each chassis. We have 4 uplink ports aggregated on each vc module. We have an additional trunked link on one module for a special purpose ESX network link (test network and vmotion on a single line).

I did put in the feature request as another poster specified. Currently if you create shared uplinks that are trunked there is no option to pass the tags thru for some of the networks. It's an either or situation. You can have your cake or you can eat it, but you can't have some networks with multiple tags and some without. You get to split the vlans off onto virtual networks, but can't pass them all thru to ESX. I think HP has heard about this and I hope this feature is forthcoming.

You can pass all vlans thru, but then you have to make sure the non ESX servers can peel the right vlan traffic off and you are dependent on the HP network software and admin configuration to do the right thing. Currently it's easier to separate trunked and non-trunked onto separate uplinks.

Other than this pretty obvious oversight in the base functionality it's easy to use and we have not had a hitch in the networking so far.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=- http://blog.mr-vm.com http://www.vmprofessional.com -=-=-=-=-=-=-=-=-=-=-=-=-=-=-
0 Kudos
vmrulz
Hot Shot
Hot Shot

Jae, good information and thanks for replying.

If I'm understanding you right we cannot pass tagged traffic through to the ESX host? Essentially we'd have to have a seperate uplink for every VLAN used on ESX? If so that would definately not work for us as we configure anywhere from 2 to 20 VLAN port groups on our ESX hosts depending on what environment they serve.

I hope HP is listening.. I'll have to ping a few HP guys I know.

0 Kudos
spex
Expert
Expert

The problem is not related to the uplinks.

The issue is, that every io/port per blade goes over the backplane to a different bay in the backend. So you have to fill every backend bay that has an active i/o port with a switch.

Also there is a difference if you use a 4 nic mezzanine card together with a san card in bl460. whether you use the bl460 in the upper or in the lower slot you do not have all ports available.

(so I was told by a hp engineer)

Regards Spex

0 Kudos
vmrulz
Hot Shot
Hot Shot

Spex, true there is a physical port mapping through to the midplane of the enclosure. However the virtual connect manager allows you to create server profiles and apply them to any blade in the enclosure. You do not need a switch in every backside slot.. that would be counter productive and expensive.

Also VC is not a true edge switch like the other non VC switches available for the c Class and is much easier to introduce to your network (no spanning tree loops).

Jae I confirmed with an HP engineer of mutual aquaintance that HP is working on a feature set upgrade that would allow for a single SUS to pass or strip VLAN tags depending on the host needs.. that will be nice!

0 Kudos
spex
Expert
Expert

Spex, true there is a physical port mapping through

to the midplane of the enclosure. However the virtual

connect manager allows you to create server profiles

and apply them to any blade in the enclosure.

How can this profiles work if the vc switch has no connectivity to the blade?

There is no crossbare in the backplane that does some magic

You do

not need a switch in every backside slot.. that would

be counter productive and expensive.

Are you still sure?

I'm not

Regards Spex

0 Kudos
vmrulz
Hot Shot
Hot Shot

Spex,

As I mentioned earlier we're sandboxing a c Class right now with two VC Ethernet modules and 2 VC FC modules. I don't know the internal workings of the enclosure and how VC interacts with it, but I can guarantee you that you can do this.

Have you got a c Class that you're working with or just researching?

The Virtual Connect User Guide may be of help.

http://h20000.www2.hp.com/bc/docs/support/SupportManual/c00865618/c00865618.pdf

Jae, sounds like that new feature set won't be released until early next year.. but it just means a few more uplinks for now.

0 Kudos
spex
Expert
Expert

We are sizing a setup together with HP for deployment...

Yes I know the vc guide. Will post again, if I can get somehow clearer - will reask @HP

BTW - how many nics you are using in your blades (2 onboard + 2 extra which BLxxx)?

Kind regards

Spex

0 Kudos
vmrulz
Hot Shot
Hot Shot

Yikes we just had a power outage.. luckily our UPS's worked this time!

Anyway, we are running bl465 and 685's right now with default configs 2 and 4 embedded NIC's respectively (no extra NIC mezz cards).

Will be interesting to see a quad core 685.. up to 128 logical processors in 10U of space.. of course you can get that with the quad core Intel bl480 now but we like AMD's architecture better.. I'd better not open that can-o-worms. Smiley Happy

0 Kudos
Jae_Ellers
Virtuoso
Virtuoso

Here's what I'm trying to say. You can't have your cake and eat it too, at least with one set of uplinks.

For "normal" traffic that routes 1 vlan to one or two server ports you can have aggregated bandwidth, shared uplink sets that are trunked. You can peel vlans, one at a time, off of this trunk, and hand them out to various server ports. So for most non ESX apps you get aggregated traffic and all your vlans over a couple uplink sets.

Now if you want to route all of those vlans to a single set of ESX ports, you are out of luck unless you create another uplink set. You can pass thru all the tags, but then you can't pull single vlans off of it. So this is the drawback. You can do it all, just not with one uplink set.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=- http://blog.mr-vm.com http://www.vmprofessional.com -=-=-=-=-=-=-=-=-=-=-=-=-=-=-
0 Kudos
vmrulz
Hot Shot
Hot Shot

spex,

This stuff is still all evolving in my brain and we're using full height blades for ESX which have 4 embedded NICs so I haven't been worried about the port mapping. If you're using half height blades and need more than two nics (a mezz NIC) those ports will be mapped to slots 3 and 4 in the chassis which would require two more VC Eth modules.

So you were sort of right in the port mapping but you wouldn't need to fully populate all rear slots unless you use all four ports in mezz 2. Geeze this blade talk is like a whole new geek dictionary!

So if I was using half height blades for ESX, I'd want 4 NIC's (2 embedded and 2 on mezz1) and 2 HBA's (2 on mezz2). I'd need VC Ethernet modules in slots 1,2,3 and 4 and I'd need VC Fibre Channel modules (or FC switches) in slots 5 and 6. Using full height blades I can eliminate the VC Eth module in slots 3 and 4.

Make sense?

0 Kudos
spex
Expert
Expert

Make sense?

I would guess - but I'm not the specialist.

There are also some complications I heard if you use quad-nic mezzanin cards and half high serverblades.

Regards Spex

0 Kudos
LeaV97
Enthusiast
Enthusiast

We've had 4 enclosures with 32 480's for a couple of months. We are in the process of adding 10 more enclosures with 460's.

If you are going to assign WWN's make sure the firmware on your Emulex adapters is up to date or it won't accept the new WWN (I can't speak for the Qlogic controlers.)

If you are going to mix half and full height make sure you understand the which mezz connects to which module on the back. Some of our new enclosures came in with Ethernet in 1, 2, 5, 6 and fiber in 3 and 4. That doesn't work if you have full high blades with a 4 port nic in mezz 2 and 2 port fiber in mezz 1.

Make sure you update the enclosure firmware.

Our 480's for ESX have a 2 port fiber in mezz 1 and a 4 port nic in mezz 2. If we add a nic to mezz 3 the most we could utilize would be a 2 port because of the way the mezz's connect to the virtual connect modules.

I'm looking forward to HP adding features that would allow one cable group to do both tagged and untagged traffic.

0 Kudos
HBBC
Contributor
Contributor

I wanted to get my head around the VLAN side of things as I don't quite fully understand it.

We have both BL460's & BL480s, the 480s being VMware hosts. We also have VC-Ethernet in Bays 1 & 2 with VC-FC in 2 & 3, each blade has factory ethernet and QLogic HBAs.

I currently have 4 ports each on the two VC-Enet modules in an LACP trunk connected to a (different each side) Allied Telesyn (Telesis) AT-9424 switch. My reading on this switch indicates that ports in an LACP trunk can only be a member of one VLAN, and all ports must be configured within the same VLAN.

So, given all this, should I be able to create a portgroup with a VLAN ID and "pass-through" these ports? Do the ports on the pSwitch not \*have* to be tagged members of said VLANs?

Or, do just the uplink ports of the pSwitch need to be tagged members?

I don't understand the "limitations" being discussed in the above thread to realise whether or not they apply to me.

(I'm on a sharp learning curve with VLANs at the moment!)

Regards,

Paul

0 Kudos