Network segmentation

[rlivermore]

I need help matching the vSwitches (to ensure failover) on our 3 hosts that reside in a vSphere 4.1 cluster and each host has 8 physical NICs. Each host has 5 vSwitches configured with 4 of them being setup identically, see below.

vSwitch0

VM port group

VMkernal port

vmnic 0 & 6

vSwitch1

VM port group

iSCSI VM network

VMkernel for iSCSI

vmnic 1 & 4

vSwitch2

VMkernel for vMotion

vmnic 2 & 5

vSwitch4

VM port group

vmnic 7

But here is where I'm hoping to get some assistance, hosts A and B have an additional vSwitch3 which are configured identically both in vSphere and at the physical switch. Host C however has vSwitch3 which is not configured like the others. Currently traffic is minimal on vSwitch3 on all of the hosts. Must I add an additional physical NIC in order to continue using both VLANs 999 and 300 below and ensure all VMs can failover? Or is there another way of accomplishing this?

vSwitch3 (hosts A & B)

VM port group

vmnic3

VLAN ID 999 (physical switch)

vSwitch3 (host C)

VM port group

vmnic3

VLAN ID 300 (physical switch)

[weinstein5]

No you will not need to add another physical NIC on Host C - you will have to another VM port group on vswitch3 configured as VLAN 999 - on the physical switch you will need to configure the port the physical NIC connects to as a trunk group with both VLANs defined - 

[rlivermore]

Thank you very much, I'm looking into your suggestion now and will report back. Thanks again!

[rlivermore]

So I have setup all vSwitch3 on each host identically and created a trunk port group with the 3 host ports included and setup the appropriate tagging for each VLAN on the pswitch. But when I connect more than one pport at the pswitch I loose connectivity to the devices. The vSwitches are configured like so:

vmnic3

port group: VM Network Voice VLAN - ID: 2

port group: SIP VLAN - ID: 999

On the pSwitch

Trunk name:: trk1

Type: LACP

Ifaces: F14 F17 F19

VLAN 999: UNTAGGED

VLAN 2: TAGGED

I tried removing the VLAN ID for each port group and leaving only one of them with an ID to no avail. I contacted HP support to ensure the trunk port group was setup correctly. Any ideas on how to get this working properly?

[rickardnobel · Accepted solution]

First, what NIC teaming policy do you use on the vSwitch?

Then, did you create the trunks on the physical switch now? One very confusing fact is that "trunk" is the Cisco name for VLAN tagging, but on your HP switches a "trunk" is link aggregation, comparable to Cisco Etherchannel.

If you want to use a HP trunk you must also have the IP Hash NIC teaming policy on the vSwitches, but you must also change your physical switch trunk setup from dynamic (LACP) to static ("HP Trunk mode"). However, it is in my opinion often best to not use these link aggregation modes at all.

[rlivermore]

I set the vSwitch teaming policy to "route based on IP hash" and set the trunk type to TRUNK as "none" is not an option and its working now. Thank you.

[rickardnobel]

Nice! And for the VLANs, did you get those correct?

The principle is that all VLAN ids that you specify on a portgroup must be includes as "tagged" on the physical switch.

[rlivermore]

I did set one of the VLAN's to untagged in the port group as "untagged" are you saying that all VLAN's must be set as "tagged"?

[rickardnobel]

No, you could use untagged too. All portgroups on the vSwitch that has no VLAN id will be "collected" into the untagged VLAN on the physical switch.

However, I do personally prefer to use only tagged VLANs, since this makes all VLAN id:s visible on the vSwitch portgroup settings. There are a somewhat higher risk of confusion if you have to remember that everything without VLAN id on the vSwitch goes into some invisible VLAN on the outside.

[rlivermore]

Argghhh, looks like I spoke too soon. I thought I had tested connecting all 3 ports into the switch but apparently I hadn't because as soon as I plug in more than one patch cable into one of the three ports in the group it breaks network connectivity.

[rickardnobel]

Could you provide screenshots of the Networking configuration + the NIC Teaming tab?

And if possible the "show run" output from the physical switch? (Passwords and other removed of course.)

[rlivermore]

Here you go...

[rickardnobel]

Could you check this post and enable CDP on all of your vSwitches: http://rickardnobel.se/archives/1110

and then connect all cables like you think they should be and then on your physical switch run:

show lldp info remote

and post the result. This is to verify how the physical switch ports really map to your vmnics.

Also, if possible, a screenshot of your other vSwitches? It was vSwitch3 on the image, are there a vSwitch0, 1 and 2?

[rlivermore]

The the hosts of interest are listed below minus joshua which does not have a guest on vSwitch3 therefore no stats.

D9        | solomon          vmnic0
  D19       | 192.168.11.194            08 ... LAN port  mobDN 7236,MITEL 53...
  E1        | 192.168.11.80             08 ... LAN port  mobDN 7216,MITEL 53...
  E2        | 20 4e 7f 73 93 64         g1
  E19       | 00 a0 c8 5c 31 dd         eth... eth 0/... TA908e
  F14       | solomon          vmnic3
  F17       | samuel           vmnic3
  F18       | 68 b5 99 a4 b7 c0         1      1         Stratus1
  G3        | 192.168.11.56             08 ... LAN port  regDN 7297,MITEL 53...
  G4        | 192.168.11.158            08 ... LAN port  regDN 5255,MITEL 53...
  G14       | solomon          vmnic6

[rickardnobel]

rlivermore wrote:

...

  F14       | solomon          vmnic3

  F17       | samuel           vmnic3

...

Solomon and Samual are two different ESXi hosts?

From your physical switch configuration:

trunk F14,F17,F19 Trk1 Trunk

This means that the Link Aggregation (Trk1) goes into two different ESXi hosts, which can never work.

If you want to use the IP hash NIC teaming policy you must create such Link Aggregation Groups on the physical switch (called "Trunks" on HP) that contains physical switch ports that goes into vmnics that are attached to both the same ESXi hosts and the same vSwitch.

[rlivermore]

So back to my original post/question: is it possible to have a vSwitch configured to connect to multiple VLAN's on multiple hosts, for host failover purposes? In order to ensure complete failover each host much have matching vSwitch configurations.

[rickardnobel]

rlivermore wrote:

So back to my original post/question: is it possible to have a vSwitch configured to connect to multiple VLAN's on multiple hosts, for host failover purposes? In order to ensure complete failover each host much have matching vSwitch configurations.

You could have multiple hosts connected to the same physical switch and using the same VLANs, but you could not have a single Link Aggregation on the physical switch going into two different hosts. This will actually be impossible for any frame to be reliable delivered.

You must break up the HP trunk and just have it going into one host. Best is, in my opinion, to not use "trunks" / IP Hash at all, and revert to Port ID.

[rlivermore]

I took the ports out of the trunk port on the ProCurve and simply set the appropriate tagging for each port in the appropriate VLAN and it seems like all is well now. Thanks for your help!

[rickardnobel]

Nice. And if you like, and have physical NIC ports, you could connect several cables into the ESXi host for redundancy. Just leave it to Port ID and make sure the correct VLANs are tagged on the switch ports.

[rlivermore]

Understood, thanks. Ofcourse we could add more in the future but as of right now I'm out of physical nics on the Hosts which is why I had to put both guests on the same vSwitch.