8 Replies Latest reply on Jan 26, 2013 10:25 AM by yelbaglff

    UCS vNICS for ESXi Management and vMotion networks/vSwitches

    MikeErter Enthusiast

      Hi UCS Community!

       

      I've heard from Cisco that using a single vNIC for vMotion and Management AND having the "Enable Failover" checkbox checked in UCSM, so UCS manages the redundant links through Fabric A and B is the way to go.

       

      And then, for Fibre Channel, iSCSI and guest networking, presenting two vNICS to ESXi- one pinned to Fabric A and the other pinned to Fabric B (no failover on them)- and letting ESXi manage the failover for those.

       

      Is this a practice UCS / ESXi shops are following?

        • 1. Re: UCS vNICS for ESXi Management and vMotion networks/vSwitches
          RaZaKKaZaR Enthusiast

          I haven't seen customers enable fabric failover and vMotion, but it sounds like an interesting design.  Was it CSE or Advanced Services Architect who recommended it?  It might be a new practice, but I don't see anything wrong with it.

           

          Whatever you decide, as long as the VM traffic is on separate vNICs, instead of using fabric failover, that will save the Fabric Interconnects from having to process the gratuitous ARPs necessary for the VMs if the vNIC does fail.

          1 person found this helpful
          • 2. Re: UCS vNICS for ESXi Management and vMotion networks/vSwitches
            MikeErter Enthusiast

            Our sales engineer, a Cisco systems engineer, suggested it as a good practice... just for the ESXi Management and vMotion networks. Thanks for the feedback!

            • 3. Re: UCS vNICS for ESXi Management and vMotion networks/vSwitches
              yelbaglff Novice

              A few items to keep in mind:

               

              • Where do I want the intelligence handled?  (FI's or ESXi)  Either/Or but not both.
              • Still follow best practices around keeping traffic separated and on different vNICs.
              • Remember that the FI's are forwarding on the data plane up both paths A and B.  With fabric failover, you will only utilize one side A or B at a time.

               

              Lastly, test different failure scenarios whether it be upstream from the FI's or downstream, but ensure failover is working no matter the failure point.  In short, both designs are valid, and I've seen both in production.

              • 4. Re: UCS vNICS for ESXi Management and vMotion networks/vSwitches
                michaelstump Enthusiast

                I haven't seen that exact design in the field. Rather, I've had success with presenting two vNICs to each ESXi service profile, marking vNIC 1 for Fabric A and vNIC 2 for Fabric B, and using QoS on a 1000v to manage traffic during congestion. So a total of 2 vNICs for ESXi that carry all of your VLANs.

                 

                The single vNIC sounds risky to me. I believe that a vNIC is associated with a backplane port on an IOM module. So a single vNIC would fail if its IOM fails. Two vNICs, one pinned to each fabric, which in effect forces them to use different IOMs, would be better in that case.

                 

                mike

                • 5. Re: UCS vNICS for ESXi Management and vMotion networks/vSwitches
                  RaZaKKaZaR Enthusiast

                  Hello Mike,

                   

                  Your design scenario is the way most UCS customers using the Nexus 1000V have their environments deployed.

                   

                  The OP is considering using fabric failover with that single vNIC just for the management and vMotion traffic.

                   

                  I am not sure why his Cisco CSE believes that approach should be recommended, but it is a valid design.  The single vNIC would still have uptime in case of single fabric failure with UCS fabric failover enabled on the vNIC.

                   

                  Regards,

                  Trevor

                  @VMTrooper

                  • 6. Re: UCS vNICS for ESXi Management and vMotion networks/vSwitches
                    balachrist Lurker

                    Dear Mail Sender,

                    Thank you for your message. Sorry I'm not in the Office until  25th of Jan 2013 till 11:00 PM IST and I have no access to my Emails.

                     

                    Incase of Urgency reach me @ 9840443663

                     

                     

                    =====

                    • 7. Re: UCS vNICS for ESXi Management and vMotion networks/vSwitches
                      michaelstump Enthusiast

                      Agreed that it should work, but I'd prefer to enable and configure failover behavior in ESXi rather than UCS. In my research on this issue, I found a thread over on the Cisco UCS forum that has a clear statement regarding this configuration: https://supportforums.cisco.com/thread/2187653.

                       

                      "Never use fabric failover with a hypervisor vswitch. Let the failover mechanism of the vSwitch handle the network outage."

                       

                      So while it may be a valid design, it looks like the consensus is to let your vSwitch handle the failover.

                       

                      mike

                      • 8. Re: UCS vNICS for ESXi Management and vMotion networks/vSwitches
                        yelbaglff Novice

                        I too have seen more folks deploying in this consensus manner, and who am I to disagree with the majority here.  But the fact remains that when using an active/standby approach, what are the pros and cons of allowing UCS fabric failover to handle this versus the hypervisor?  Is it comfort and familiarity or are there truly perceived or actual disadvantages when using fabric failover in an active/standby deployment instead of letting the hypervisor handle this?

                         

                        Also, there are some very valid use cases for fabric failover like hypervisor pass-through for example.