LACP and Brocade connections to VSAN

[WhiskyTangoFoxt]

Good afternoon,

I'm a little confused about the best way to connect our storage switches to our hosts for VSAN.

We have 4x Dell R730xd with 2 disk groups in each. I have them connected to a pair of Brocade VDX-6720s that are joined by ISL fabric.

Currently The dedicated VSAN DV Switch is configured with both VDX1 and VDX2 set as active uplinks, routing based on originating virtual port.I have the VLAN set at the switch, untagged across the 4 ports on each switch.

However I have two documents that seem to configure it different ways; the Brocade-EVO Rail guide tagging the ports as individuals:

"20. Configure the server-facing network interfaces

     Configure the ports on the switch which connect to the EVO:RAIL nodes

interface tengig 1/0/1-4

Next put the port into trunk mode by entering the following two commands:

switchport

switchport trunk mode"

No mention of LAG except for edge ports connecting the Brocade back to the main LAN.

Or the VSAN Network Design guide referencing Architecture 2 - Stacked Switches indicating that the two switches should have their common host ports joined in a LAG:

"Table 5 lists network adapter teaming policies supported when switches are stacked. To achieve bandwidth

aggregation, the “Route based on IP hash” policy with all adapters’ being active should be set in the distributed

port group for Virtual SAN and other port groups that share the same uplinks. All other policies can result in load

sharing, but not load balancing, regardless of whether the teaming is in active–standby or active–active mode,

as with architecture 1.

With IP hash-based load-balancing policy, all physical switch ports connected to the active vmnic uplinks must

be configured with static EtherChannel or LACP. This ensures that the same hash algorithm is used for traffic

returning in the opposite direction. And IP hash-based load balancing should be set for all port groups using the

same set of uplinks. All vmnics in the team can be used for Virtual SAN traffic.

So does this mean that the EVO-Rail design guide will result in load sharing, rather than load balancing?

I'd like to do it right the first time.

Brad

[jonretting]

Personally I do not recommend LAG/LACP for VSAN traffic. After playing around extensively with multiple environments, including the lab, LACP seems untenable on VSAN. I have yet to ever see proper aggregation of traffic is any form. VSAN seems to just use whatever the link speed is, regardless of hash policy or LACP config. Moreover in these cases the VSAN network was unstable. Nearly always I opt for explicit failover for VSAN, more then one uplink is not possible without LACP, so "Routing based on anything" is not possible, or so i have come to think. Last I read beacon is not functional on VSAN, but that may have changed. I cannot comment as to using redundant switching and how it applies to VSAN and LACP. Granted this is a 2013 post http://www.yellow-bricks.com/2013/10/29/virtual-san-network-io-control/ but has served me well for best practices in production with 2x 10GB ports. Thanks, -Jon

[zdickinson]

I did not see any stability issues with LAGs, but did find them complicated with just 3 hosts.  Our final config was just using individual ports with local host virtual switches.  If we had a bunch of hosts I would go with VDS and LAGs.

In theory I think you're right about the load balancing and load sharing.  However, with VDS and LAGs I did not see any load balancing.  It just used one link.

Thank you, Zach.

[JohnNicholson]

As weird as this is, those are the exact same switches I have in our VSAN lab. (REALLY nice switches, great NSX support).

As far as failover for VSAN remember best practice is to deploy 2 VSAN networks with different subnet's VLAN's etc, so what we do is have a VSAN A network (on one switch) and a VSAN B network (on the other switch).

[jonretting]

Wanted to add...  This setup employes two VMK (VLAN A/B) adapters for VSAN per node, in different Port Groups (VLAN A/B), and assigned its proper switch uplink in failover. Off hand I think that's correct.  Do you know if beacon is supported for a VSAN uplink now? Best, -Jon