4 Replies Latest reply on Aug 9, 2015 10:09 PM by jonretting

    LACP and Brocade connections to VSAN

    WhiskyTangoFoxtrot Novice

      Good afternoon,


      I'm a little confused about the best way to connect our storage switches to our hosts for VSAN.

      We have 4x Dell R730xd with 2 disk groups in each. I have them connected to a pair of Brocade VDX-6720s that are joined by ISL fabric.

      Currently The dedicated VSAN DV Switch is configured with both VDX1 and VDX2 set as active uplinks, routing based on originating virtual port.I have the VLAN set at the switch, untagged across the 4 ports on each switch.


      However I have two documents that seem to configure it different ways; the Brocade-EVO Rail guide tagging the ports as individuals:

      "20. Configure the server-facing network interfaces

           Configure the ports on the switch which connect to the EVO:RAIL nodes

      interface tengig 1/0/1-4

      Next put the port into trunk mode by entering the following two commands:


      switchport trunk mode"

      No mention of LAG except for edge ports connecting the Brocade back to the main LAN.

      Or the VSAN Network Design guide referencing Architecture 2 - Stacked Switches indicating that the two switches should have their common host ports joined in a LAG:


      "Table 5 lists network adapter teaming policies supported when switches are stacked. To achieve bandwidth

      aggregation, the “Route based on IP hash” policy with all adapters’ being active should be set in the distributed

      port group for Virtual SAN and other port groups that share the same uplinks. All other policies can result in load

      sharing, but not load balancing, regardless of whether the teaming is in active–standby or active–active mode,

      as with architecture 1.

      With IP hash-based load-balancing policy, all physical switch ports connected to the active vmnic uplinks must

      be configured with static EtherChannel or LACP. This ensures that the same hash algorithm is used for traffic

      returning in the opposite direction. And IP hash-based load balancing should be set for all port groups using the

      same set of uplinks. All vmnics in the team can be used for Virtual SAN traffic.

      So does this mean that the EVO-Rail design guide will result in load sharing, rather than load balancing?

      I'd like to do it right the first time.


        • 1. Re: LACP and Brocade connections to VSAN
          jonretting Enthusiast

          Personally I do not recommend LAG/LACP for VSAN traffic. After playing around extensively with multiple environments, including the lab, LACP seems untenable on VSAN. I have yet to ever see proper aggregation of traffic is any form. VSAN seems to just use whatever the link speed is, regardless of hash policy or LACP config. Moreover in these cases the VSAN network was unstable. Nearly always I opt for explicit failover for VSAN, more then one uplink is not possible without LACP, so "Routing based on anything" is not possible, or so i have come to think. Last I read beacon is not functional on VSAN, but that may have changed. I cannot comment as to using redundant switching and how it applies to VSAN and LACP. Granted this is a 2013 post http://www.yellow-bricks.com/2013/10/29/virtual-san-network-io-control/ but has served me well for best practices in production with 2x 10GB ports. Thanks, -Jon

          • 2. Re: LACP and Brocade connections to VSAN
            zdickinson Expert

            I did not see any stability issues with LAGs, but did find them complicated with just 3 hosts.  Our final config was just using individual ports with local host virtual switches.  If we had a bunch of hosts I would go with VDS and LAGs.


            In theory I think you're right about the load balancing and load sharing.  However, with VDS and LAGs I did not see any load balancing.  It just used one link.


            Thank you, Zach.

            • 3. Re: LACP and Brocade connections to VSAN
              JohnNicholson Enthusiast

              As weird as this is, those are the exact same switches I have in our VSAN lab. (REALLY nice switches, great NSX support).

              As far as failover for VSAN remember best practice is to deploy 2 VSAN networks with different subnet's VLAN's etc, so what we do is have a VSAN A network (on one switch) and a VSAN B network (on the other switch).

              • 4. Re: LACP and Brocade connections to VSAN
                jonretting Enthusiast

                Wanted to add...  This setup employes two VMK (VLAN A/B) adapters for VSAN per node, in different Port Groups (VLAN A/B), and assigned its proper switch uplink in failover. Off hand I think that's correct.  Do you know if beacon is supported for a VSAN uplink now? Best, -Jon