vSwitch and VLAN tag problems

[royiii]

I have a new server with 6 NICS.

I'm trying to configure them the best way. My ideas for configuration are:

1.Dedicate NIC0 to the server console

2.Dedicate NIC2 and NIC4 to vSwitch1 and adding my VLAN tags(4) to this switch

3.Dedicate NIC1 and NIC3 to vSwtich2 and adding my VLAN tags(4) to this switch also.

I'm confused and having a problem when configuring these NICS and vSwitches and connecting them to an Extreme switch.

The problem is that I am not able to properly configure the vSwitch(es) and get things working. In testing I have added vlan tags to my service console nic and things appear to work.

To keep the diagnostics a little simpler I'm only troubleshooting with vSwitch1. Here I have two nics and I plug them into an Extreme Summit 400.

Do these two ports have to be "Shared"/Bonded/trunked? When I do configure the two Summit ports as shared I am not getting anywhere.

Any ideas or suggestions are greatly appreciated since I have been stuck on this for a number of days.

Below is a copy of my latest esxcfg-vswtich -l

# esxcfg-vswitch -l

Switch Name Num Ports Used Ports Configured Ports Uplinks

vSwitch0 32 6 32 vmnic0

PortGroup Name Internal ID VLAN ID Used Ports Uplinks

900esxprivate portgroup8 900 0 vmnic0

300prteni portgroup6 300 0 vmnic0

200mgmteni portgroup5 200 0 vmnic0

100enihouston portgroup4 100 1 vmnic0

500dhcpeni portgroup7 500 1 vmnic0

Service Console portgroup0 0 1 vmnic0

800vmotion portgroup3 800 1 vmnic0

Switch Name Num Ports Used Ports Configured Ports Uplinks

vSwitch1 64 3 64 vmnic2,vmnic4

PortGroup Name Internal ID VLAN ID Used Ports Uplinks

100vSwitch1 portgroup10 100 0 vmnic4,vmnic2

500vSwitch1 portgroup14 500 0 vmnic4,vmnic2

300vSwitch1 portgroup13 300 0 vmnic4,vmnic2

200vSwitch1 portgroup12 200 0 vmnic4,vmnic2

Switch Name Num Ports Used Ports Configured Ports Uplinks

vSwitch2 64 3 64 vmnic5,vmnic3

PortGroup Name Internal ID VLAN ID Used Ports Uplinks

100vSwitch2 portgroup11 100 0 vmnic3,vmnic5

[chouse]

I think you need to configure the physical switch ports as trunked so they pass all vlans. Also, if you have 6 NICs, you should have 2 nics on a vSwitch with Service Console and VMkernel port groups, 2 nics on another vswitch, and the last 2 nics on the last vswitch. You don't want to leave any vswitch without dual uplinks, especially the service console.

[bggb29]

You need to configure the pswitch as a dot1q trunk.

You listed 5 nics, So I assume the 6 will be for vmotion,

Are you planning on HA and DRS you will want to consider

redundancy for vswf0 even if it is to have the vswitch with vmotion

setup as a failover.

[Gabrie1]

If nic1, 2, 3 and 4 will have the same VLANs I would create 1 vSwitch and connect 4 pNics to it. Link 2 to physical switch nr1 and 2 to nr2.

Gabrie

[royiii]

That is the way that I had configured my previous servers(4 nics bonded to one vswitch), but was under the impression that I might be able to balance performance by splitting the 4 NIC's into two seperate vswitches.

I am still not getting the physical switch to respond properly after setting up the trunking. I have spoken with Extreme and not yet gotten resolution.

Do I have to assign an IP address to each bond? I didn't think so but this problem is still holding me up.

Does anyone have a sample configuration with Extreme switches? Just in case there is an Extreme person out there I have attached my switch config.

Thanks

[bggb29]

You do not need to setup a ip address on the bond on the vswitch side. The vswitch will tag the packet with a vlan tag.

You should have a interface on the pswitch or router that has a ip address for the gateway.

You can balance performance using different hashing on the way out of esx i/e mac in mac out etc.

Sorry I have no knowledge of extreme switches at all.

[Gabrie1]

Hi

Esx won't load balance between vswitches. It will however do some sort of balance within one vSwitch over the physical nics within that switch. If you have 4 nics of 1Gb in one vswitch, each connection between VM and external server / pc / VM is still max 1Gb.

What load balance is concerned, there are 3 balance options, none of them is a load balance. The new default is VirtualPortID, which makes a VM-1 always take to physical nic1 in that vSwitch, untill nic1is no longer available and the connection will then failover to the next available nic. VM2 wil always talk to nic2, VM3 to nic3, VM4 to nic1, VM5 to nic2, VM6 etc etc. This is the prefered balance method by ESX3.

Next is IP-out. This makes a VM use different physical nics based on a hash generated out of the destination IP address of the packets. This means that VM1 will talke to client1 over nic1 and probably talk to client2 over nic2. This option however can cause problems on your physical switches because now the mac address of VM1 flows over nic1 and over nic2. Your physical switch might become confused about this and will cause hickups. I've forgotten the exact term, but you need special configuration on your physical switches for this. It is only recommended if balancing by VirtualPortID realy isn't working for you.

Third balance method is based on Mac-out. This is the old way that used to be in ESX2. It is now only her for compatibility reasons. In Nice (Fr) at TSX event, there was a session about networking, in which the VMware engineer advised not to use mac-out any more. Mac-out is almost the same as virtualPortId. Now the outgoing nic is determined based on a hash generated on the mac address of the VMs nic. VirtualPortID does almost the same but is much faster, because it doesn't calculate any hashes.

Regarding your other question:

- An esx physical nic or bond, never has an Ip address.

- On cisco switches you have to enable portfast and use 802.1q trunks

Gabrie