11 Replies Latest reply on Apr 21, 2009 12:08 AM by mreferre

    The Great vSwitch Debate

    Ken.Cline Champion

       

      A question that seems to come up quite frequently here on the forums is "How should I configure my vSwitch?". I've started a series of posts over on my blog that, I hope, will help to answer this question. The first four posts in the series are dealing with defining what a vSwitch is and how it operates. My next post (Part 5) will begin a discussion of how I recommend configuring vSwitches with various numbers of pNICs. I'm hoping to get some good discussion going to either agree with me or tell me that I'm an idiot and don't have a clue what I'm talking about. Either way, your comments are welcome - and I hope at least some of your questions about virtual switches will be answered!

      </shameless self promotion>

       

      Ken Cline

      VMware vExpert 2009

      VMware Communities User Moderator

      Blogging at: http://KensVirtualReality.wordpress.com/

        • 1. Re: The Great vSwitch Debate
          azn2kew Champion

           

          Ken,

           

           

          Those are great articles and it really benefit for everyone who needs to learn about vNetworking and vSwitches for their data center designs.  Now let's throw out some scenarios here so someone can post/replies.

           

           

          1. What is the minimum pNICs that generally work for small/medium environment?

          2. What is the standard pNICs use for large enterprise environment which includes (Production, Test, Dev, DMZ, SC, VMotion, NFS, iSCSI, Backups)

          3. In what circumstances we have to use VLANs and are their advantages and disadvantages for vLAN comparing to standard pSwitch ports/pNICs connection.

          4. What scenarios we need to team 4 pNICs?  We all run 2 pNIC team and it works fine in most environment right?

           

          I've also seen good articles from Edward Haletky on combination of pNICs scenarios such as 2 pNICs up to 8 pNICs.  Generally speaking, 4 pNICs would do the trick in most small/medium environments, but if it requires more complex networking, then it would requires 6-10 pNICs for each port groups and redundancies as mentioned on your articles. 

           

           

          For my standard build especially financial and government clients, their have strict performance and security requirements and complex networks, I've always used 10 pNICs for all my standard ESX 3.5 builds 2 - built in NICs plus 2 x quad ports Intel that does all the networking piece I've need.  If I do not use up all 10 pNICs, it could use it for spare in case it failed and I just have to link it and its online without swap the adapter requires power down for any hardware replacement. 

           

          If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!!

           

          Regards,

           

          Stefan Nguyen

          VMware vExpert 2009

          iGeek Systems Inc.

          VMware, Citrix, Microsoft Consultant

           

           

          1 person found this helpful
          • 2. Re: The Great vSwitch Debate
            Ken.Cline Champion

            Thanks, Stefan,

             

            I'll try to hit all the questions you've posted as well as any others that may show up in this thread. It's always better to have multiple minds working on something!

             

            Ken Cline

            VMware vExpert 2009

            VMware Communities User Moderator

            Blogging at: http://KensVirtualReality.wordpress.com/

            • 3. Re: The Great vSwitch Debate
              athlon_crazy Master

               

              I went tru your articles (part 1 & 2) and I admit, it's good article indeed. Good explanation, good example and easy to understand for noobie like me with a lack networking experience.

               

               

               

               

               

              BTW, just want to ask if you dont mind, since vNIC teaming  can be done for 2 purposes :

               

              • Fault tolerence

              • Load balance

               

              I dont have a problem with both setup, the only question is, why my load balance for VMs traffic working only with "Route based on Source Mac Hash"  instead "Route based on originating virtual port ID" which is recommended one? FYI, my setup without Layer 3 pSwitch involved. 

               

               

               

               

               

              System Engineer

              Zen Systems Sdn Bhd

              Malaysia

              www.no-x.org

               

               

              • 4. Re: The Great vSwitch Debate
                TomHowarth Guru User Moderators vExpert

                If the vSwitch configuration discussion is half as good as your positioning blogs then this will be a  series to remember.  Keep up the good work Ken

                 

                 

                 

                 

                If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

                 

                Tom Howarth VCP / vExpert

                VMware Communities User Moderator

                Blog: www.planetvm.net

                Contributing author for the upcoming book "VMware Virtual Infrastructure Security: Securing ESX and the Virtual Environment”.

                 

                 

                • 5. Re: The Great vSwitch Debate
                  Ken.Cline Champion
                  athlon_crazy wrote:

                  I dont have a problem with both setup, the only question is, why my load balance for VMs traffic working only with "Route based on Source Mac Hash"  instead "Route based on originating virtual port ID" which is recommended one? FYI, my setup without Layer 3 pSwitch involved. 

                   

                  Is your problem similar to this one? If so, please jump in on that thread with additional details about your configuration. Let's try to keep that discussion together in one place!

                   

                  Ken Cline

                  VMware vExpert 2009

                  VMware Communities User Moderator

                  Blogging at: http://KensVirtualReality.wordpress.com/

                  • 6. Re: The Great vSwitch Debate
                    mreferre Virtuoso

                    Ken,

                     

                    great articles.

                     

                    Warning: arguing about the right number of pNICs (and their configs) might be as difficult as arguing scale up vs scale out or blonds vs brunettes......

                     

                    Massimo.

                    • 7. Re: The Great vSwitch Debate
                      Ken.Cline Champion
                      king@it.ibm.com wrote:

                      Ken,

                       

                      great articles.

                       

                      Warning: arguing about the right number of pNICs (and their configs) might be as difficult as arguing scale up vs scale out or blonds vs brunettes......

                       

                      Massimo.

                       

                      That's a good thing for a blog ... drive traffic

                       

                      I don't plan to recommend the "right" number of pNICs, more along the lines of "If you have 4 pNICs, here's a good config"...

                       

                      Ken Cline

                      VMware vExpert 2009

                      VMware Communities User Moderator

                      Blogging at: http://KensVirtualReality.wordpress.com/

                      • 8. Re: The Great vSwitch Debate
                        Braumin Enthusiast

                        As someone relatively new to the Virtualization scene I have to say that I loved that blog post.  It really made the whole Virtual Switch thing make sense for me.  Thanks for the great work!

                        • 9. Re: The Great vSwitch Debate
                          RobVM Novice

                          Ken,

                           

                          I am eagerly anticipating part 5 so the debate can begin! I have a couple of lingering questions related to COS/vMotion and iSCSI, and I'm hoping by the time the smoke has cleared I have an answer. My first question was, with 8 pNICs, I have seen recommendations that show 2 pNIC for COS and 2 for vMotion, and other setups where COS and vMotion shared 2 pNICs in two vSwitches. It seems to me that 2 pNICs would work fine for this, but others might say it is not as secure or redundant. My second lingering question regards iSCSI and Guest OS connections. Let's say I have a guest(s) I want to direct connect to iSCSI volumes for higher throughput or MPIO, and I have a seperate physical network (dedicated switches) for my iSCSI traffic. Would the guest use the VMkernel vSwitch that ESX uses to connect to the iSCSI, or, would I dedicate 2 more pNICs to iSCSI for the guests (for a total of 4 pNICS dedicated to iSCSI out of 8 possible)? I'd be interested to hear your thoughts on either subject.

                           

                           

                          Regards,

                           

                           

                          -Rob

                           

                           

                          Update: OK, I've answered part of the second question. iSCSI connections in ESX appear as standard SCSI connections to guests. In order to let the guest OS connect using a software iSCSI initiator with MPIO to a LUN, two additional pNICs would need to be configured. I'm still interested to find out what the best vSwitch scenario would be in this circumstance considering 8 pNICs on the host.

                           

                          Message was edited by: RobVM

                          1 person found this helpful
                          • 10. Re: The Great vSwitch Debate
                            Ken.Cline Champion

                            RobVM,

                             

                            Check out the latest installment: The Great vSwitch Debate - Part 6

                             

                            I'm beginning to think I should have called this "The VMware vSwitch - the Authoritative Reference" or something...not much debating going on!

                             

                            Ken Cline

                            VMware vExpert 2009

                            VMware Communities User Moderator

                            Blogging at: http://KensVirtualReality.wordpress.com/

                            • 11. Re: The Great vSwitch Debate
                              mreferre Virtuoso

                              How can you argue with the master ?

                               

                              Massimo.

                               

                              P.S. give me credit for having tried to argue around part 2 or 3 (I don't remember).