Accepted Solutions
what happens when I:
Attach all 4 ports to the switch, create a standard vSwitch on the ESXi and attach all 4 NICs as active with LB policy "based on orginating Port ID".
Based on my understanding it will distribute all my VMs across these NICs attached and all VMs are able to communicate (send AND receive) via those 4 vNICS/NICS. It will failover a VM which is pinned to a VNIC to another VNIC when i unplug the specific wire.
BUT
My collegues told me NO! You will be only able to send data over all NICs but receive only over one vNIC due to the source/destination MAC address of the VMs respectively the NIC of the host.
Is that true? What exactelly is the process of this policy "orginating port ID". Because there is an additional one called "based on MAC....".
It's pretty straightforward, in the default "based on orginating Port ID" configuration each vNIC of a VM only has a single active physical uplink at any given point in time. This physical NIC is used by the host to send as well as receive traffic for this particular vNIC (because your physical switch will learn the VM vNIC MAC on the currently active port it will only forward traffic through this link).
The distribution is static and only changes when you add/remove/plug/unplug physical uplinks or the vNIC (i.e. power off/power on, disconnect/connect the vNIC). The behavior is the same for standard as well as distributed vSwitches with this policy. You can see the current mapping in the (r)esxtop network view:
The Route based on source MAC hash option is very similar, but instead of using the internal virtual port ID, it establishes a static mapping based on the source MAC address of an Ethernet frame being sent from a vNIC. This approach also maintains a nice static MAC table on your physical switch to prevent MAC flapping. Traffic of a single VM vNIC will only use multiple ports if the VM uses multiple source MAC addresses. Usually that should never occur unless you do some fancy networking stuff inside the VM like multiple virtual interfaces, and it also means you need to allow forged transmits and MAC changes in the vSwitch port group security options. This article sums it up nicely:
Hostile Coding: VMware: MAC hash based LB
A server has 2x quad port NICS. There is a vswitch with 2 attached NICs (first on card 1, first on 2nd one) and we configure "based on originating port ID". Further we do NOT configure a stack neither a Etherchanne on the switches which are connected to the NICs (via cross).
We only want to have simpe LB and failover in case of a path outage. Will this work properly (send and receive)???? My colleagues told me that will not work as long as the switches are not stacked?? Why should it not? From my understanding the VM will switch over to another vNIC which is connected to another NIC which again is connected to an other standalone switch.
Your colleague is wrong, you do not need any special configuration for the port ID based load balancing mechanism. This will work just fine with failover, due to the static simple mapping explained above. It will be just like plugging a system from one switch into the other (in the same broadcast domain).
When do wenn need to configure "Based on MAC" or "IP HASH"? Well i know that when you configure "IP Hast" then you need to setup an Etherchannel between the Host and the Switch and set the policy to IP Hash otherwise you dont have additional bandwith.
The MAC based forwarding is also static, but based on the source MAC and has nothing to do with etherchannel/LACP. See my explanation and the article above.
Why would we need to set adapters to standby and not active? Is that used to deny issues with loops in a not spanning-tree environment?
vSwitches won't form loops unless you do some really ugly things in a VM with multiple vNICs and configure some birding within that VM OS. The active/standby settings are basically just primary/secondary prioritization in case you want traffic of a particular port group to go through one specific uplink, unless a failover occurs.
How do you configure the network (amount of NICs, vSwitches, portgroup assignment, vmotion etc. and the switches)?
It depends. On a lot of factors.
To sum it up:
- segregate your network with VLANs as if in a completely physical setup
- put vMotion on a private non-routed VLAN with a dedicated physical uplink (or active/standby team settings that make sure vMotion is not shared with other traffic except in a failover case)
- use as many physical uplinks as you need for bandwidth
- for IP-storage (NFS, iSCSI) or FT, use dedicated physical uplinks as well
what happens when I:
Attach all 4 ports to the switch, create a standard vSwitch on the ESXi and attach all 4 NICs as active with LB policy "based on orginating Port ID".
Based on my understanding it will distribute all my VMs across these NICs attached and all VMs are able to communicate (send AND receive) via those 4 vNICS/NICS. It will failover a VM which is pinned to a VNIC to another VNIC when i unplug the specific wire.
BUT
My collegues told me NO! You will be only able to send data over all NICs but receive only over one vNIC due to the source/destination MAC address of the VMs respectively the NIC of the host.
Is that true? What exactelly is the process of this policy "orginating port ID". Because there is an additional one called "based on MAC....".
It's pretty straightforward, in the default "based on orginating Port ID" configuration each vNIC of a VM only has a single active physical uplink at any given point in time. This physical NIC is used by the host to send as well as receive traffic for this particular vNIC (because your physical switch will learn the VM vNIC MAC on the currently active port it will only forward traffic through this link).
The distribution is static and only changes when you add/remove/plug/unplug physical uplinks or the vNIC (i.e. power off/power on, disconnect/connect the vNIC). The behavior is the same for standard as well as distributed vSwitches with this policy. You can see the current mapping in the (r)esxtop network view:
The Route based on source MAC hash option is very similar, but instead of using the internal virtual port ID, it establishes a static mapping based on the source MAC address of an Ethernet frame being sent from a vNIC. This approach also maintains a nice static MAC table on your physical switch to prevent MAC flapping. Traffic of a single VM vNIC will only use multiple ports if the VM uses multiple source MAC addresses. Usually that should never occur unless you do some fancy networking stuff inside the VM like multiple virtual interfaces, and it also means you need to allow forged transmits and MAC changes in the vSwitch port group security options. This article sums it up nicely:
Hostile Coding: VMware: MAC hash based LB
A server has 2x quad port NICS. There is a vswitch with 2 attached NICs (first on card 1, first on 2nd one) and we configure "based on originating port ID". Further we do NOT configure a stack neither a Etherchanne on the switches which are connected to the NICs (via cross).
We only want to have simpe LB and failover in case of a path outage. Will this work properly (send and receive)???? My colleagues told me that will not work as long as the switches are not stacked?? Why should it not? From my understanding the VM will switch over to another vNIC which is connected to another NIC which again is connected to an other standalone switch.
Your colleague is wrong, you do not need any special configuration for the port ID based load balancing mechanism. This will work just fine with failover, due to the static simple mapping explained above. It will be just like plugging a system from one switch into the other (in the same broadcast domain).
When do wenn need to configure "Based on MAC" or "IP HASH"? Well i know that when you configure "IP Hast" then you need to setup an Etherchannel between the Host and the Switch and set the policy to IP Hash otherwise you dont have additional bandwith.
The MAC based forwarding is also static, but based on the source MAC and has nothing to do with etherchannel/LACP. See my explanation and the article above.
Why would we need to set adapters to standby and not active? Is that used to deny issues with loops in a not spanning-tree environment?
vSwitches won't form loops unless you do some really ugly things in a VM with multiple vNICs and configure some birding within that VM OS. The active/standby settings are basically just primary/secondary prioritization in case you want traffic of a particular port group to go through one specific uplink, unless a failover occurs.
How do you configure the network (amount of NICs, vSwitches, portgroup assignment, vmotion etc. and the switches)?
It depends. On a lot of factors.
To sum it up:
- segregate your network with VLANs as if in a completely physical setup
- put vMotion on a private non-routed VLAN with a dedicated physical uplink (or active/standby team settings that make sure vMotion is not shared with other traffic except in a failover case)
- use as many physical uplinks as you need for bandwidth
- for IP-storage (NFS, iSCSI) or FT, use dedicated physical uplinks as well