adamwiso
Enthusiast
Enthusiast

HP FlexFabric Use Cases VMWare Best Practice

Jump to solution

Hello all.

I am trying to validate a few configurations in a particular environment.  ESXi 5.5 - vDS - Enterprise Plus Licensing, C7000 Chasis, with BL460 G7s - in regards to the eth networking to the ESXI hosts.  Fiber Channel is sound.

Basically each chasis have a 10GB uplink to the core switch in each HP VC Flexfabric module.  All VLANS are trunked down these and the applicable VLANS are configured in the HP Virtual Connect.  My questions lie on the VMware side for the most part.  Basically each host is presented with 2 10GB nics courtesy of the HP FlexFabric configurations.  From here the last admin setup one virtual distributed switch, broke out with the applicable VDPG, tagging all the VLANS, looks good so far.  All VDPG are setup in active/active with no teaming and failover on any of them.  Also on one of the VDPG, it is setup for management and vmotion, thus 1 VLAN for both Management and vMotion (which I know if not best practice) and on the same subnet.  Which I read can cause issues (on FrankDennemans blog about Vmotion traffic going out on VMK0).  Then looking at the hosts (VMKernel Adapters) one has Vmotion enabled and the other has management traffic enabled.  Also NO NIOC is setup anywhere in this environment.

I could see why the legacy admin would want to separate the vmotion and management on separate VMK nics to not saturate the links during DRS or vMotions, however it clearly looks like a single point of failure (if a module goes down).  I looked for hours today on HP cookbooks and white documents, and nothing clearly stated this.  I want to break out the vMotion and management on separate VLANS, however not sure if I should create 2 more nics in the virtual connect and use these for vmotion (active/standby) x2 and use NIOC - or if I can somehow modify the above setup, and then enable NIOC.

I read lots of blogs on this, and I am just trying to eliminate any SPOF and stay within best practice, any help you could offer would be great.  Sorry if I missed any information.

Tags (5)
0 Kudos
1 Solution

Accepted Solutions
chriswahl
Virtuoso
Virtuoso

I could see why the legacy admin would want to separate the vmotion and management on separate VMK nics to not saturate the links during DRS or vMotions, however it clearly looks like a single point of failure (if a module goes down).  I looked for hours today on HP cookbooks and white documents, and nothing clearly stated this.  I want to break out the vMotion and management on separate VLANS, however not sure if I should create 2 more nics in the virtual connect and use these for vmotion (active/standby) x2 and use NIOC - or if I can somehow modify the above setup, and then enable NIOC.

A vmkernel interface does not represent a single point of failure, so long as the underlying port group is set up for multiple network adapters with active or standby links. If the active vmnic fails, the vmk will move to the next active vmnic or, failing that, select a standby vmnic.

So long as you have one or more vmnics from each FlexFabric module, you're achieving a high availability solution. Should the module fail, or the uplinks feeding the module fail, the underlying vmnic interface will show link down status and the hypervisor will act accordingly.

VCDX #104 (DCV, NV) ஃ WahlNetwork.com ஃ @ChrisWahl ஃ Author, Networking for VMware Administrators

View solution in original post

0 Kudos
7 Replies
chriswahl
Virtuoso
Virtuoso

I could see why the legacy admin would want to separate the vmotion and management on separate VMK nics to not saturate the links during DRS or vMotions, however it clearly looks like a single point of failure (if a module goes down).  I looked for hours today on HP cookbooks and white documents, and nothing clearly stated this.  I want to break out the vMotion and management on separate VLANS, however not sure if I should create 2 more nics in the virtual connect and use these for vmotion (active/standby) x2 and use NIOC - or if I can somehow modify the above setup, and then enable NIOC.

A vmkernel interface does not represent a single point of failure, so long as the underlying port group is set up for multiple network adapters with active or standby links. If the active vmnic fails, the vmk will move to the next active vmnic or, failing that, select a standby vmnic.

So long as you have one or more vmnics from each FlexFabric module, you're achieving a high availability solution. Should the module fail, or the uplinks feeding the module fail, the underlying vmnic interface will show link down status and the hypervisor will act accordingly.

VCDX #104 (DCV, NV) ஃ WahlNetwork.com ஃ @ChrisWahl ஃ Author, Networking for VMware Administrators
0 Kudos
homerzzz
Hot Shot
Hot Shot

Just to add to this, you can break out Management and vMotion to two separate PGs without adding additional nics to the hosts as long as your dv switch is configured with two uplinks (vmnics) already and you can also enable NIOC.

I have a similar setup to you and have never saturated the 10GB uplinks... and I use both uplinks for vmotion.

0 Kudos
adamwiso
Enthusiast
Enthusiast

Chris and homerzz - thanks for the responses (much appreciated)

Should I break out the PG for vMotion how I traditionally am familiar with in

(vmotion1) 1PG - vMotion vmnic1 active vmnic2 standby NIC

(vmotion2) 1PG -  vMotion vmnic2 active vmnic1 standby NIC

Then enable NIOC so I do not saturate the links during MM or DRS?

0 Kudos
homerzzz
Hot Shot
Hot Shot

That is exactly how I do it.

0 Kudos
chriswahl
Virtuoso
Virtuoso

Pretty much. NIOC may in fact let the links get saturated, so long as no other traffic is demanding time on the link, by way of the share values.

I would avoid setting hard limits and have written a post on dual NIC vMotion that might help.

I've also outlined a corner case where NIOC is not enough.

VCDX #104 (DCV, NV) ஃ WahlNetwork.com ஃ @ChrisWahl ஃ Author, Networking for VMware Administrators
0 Kudos
homerzzz
Hot Shot
Hot Shot

Chris certainly knows what he is talking about! Read his blogs!

0 Kudos
adamwiso
Enthusiast
Enthusiast

Chris thanks for the link - and of course I have been to your blog (you and your team keep up the great work).

Thank you both for the help!

0 Kudos