6 Replies Latest reply on Aug 16, 2020 3:29 PM by Petersaints

    NSX-T 3.0 design opinion

    Petersaints Novice

      Hello all,

      I will need to deploy NSX-T 3.0 on a shared cluster (vSphere 6.7) with 4 pnics to dedicate to NSX-T and i'm thinking on the following design:

       

       

      - 2 pnic assign to a vDS for the Edges

      - 2 pnics assign to a N-VDS

       

      My doubts are:

       

      1 - Do i have any advantage in having only one TZ for VLAN and other TZ for overlay, or have a 3rd TZ (vlan type) for the Edges?

       

      2 - With this design with a dedicated vDS for the Edges, will i need a different TEP pool for the Edges or i can use the same TEP pool that i use for the hosts?

       

      3 - If i had to use a different TEP pool for the Edges, that vlan have to be trunked on the vDS pnics or only on the N-VDS pnics?

       

      4 - What is the advantage of having a vDS for the Edges instead of an N-VDS?

       

       

      Thanks.

       

      Kind regards,

        • 1. Re: NSX-T 3.0 design opinion
          EricPedersen Lurker

          Is this a production deployment or for lab purposes? Your design choices are influenced by the underlying network. For example, we have dedicated vlan transport zones for our edge nodes that are just used for the connections to the upstream network. The edge nodes don't need to connect to our other vlan-backed segments that are used by our workload VMs that haven't been moved to an overlay network.

          • 2. Re: NSX-T 3.0 design opinion
            Petersaints Novice

            Hi,

             

            It's for production. In my case, i was thinking in use the edge's vlan's just to N/S.

            • 3. Re: NSX-T 3.0 design opinion
              Sreec Master
              vExpertVMware EmployeesCommunity Warriors

              Any specific reason why you are opting for NVDS based approach ?  I would highly recommend to go with VDS 7.0 ( if it is feasible to upgrade vsphere to 7.0) based integration since it is fully supported design.

               

              1 - Do i have any advantage in having only one TZ for VLAN and other TZ for overlay, or have a 3rd TZ (vlan type) for the Edges?

               

              VLAN based TZ is for what purpose ? Is it just for connecting VLAN backed networks ?

               

              2 - With this design with a dedicated vDS for the Edges, will i need a different TEP pool for the Edges or i can use the same TEP pool that i use for the hosts?

               

              3 - If i had to use a different TEP pool for the Edges, that vlan have to be trunked on the vDS pnics or only on the N-VDS pnics?

               

              4 - What is the advantage of having a vDS for the Edges instead of an N-VDS?

               

               

              NSX Edge VM can be installed on an NSX-T Data Center prepared host and configured as a transport node. There are two types of deployment:

              • NSX Edge VM can be deployed using VSS/VDS port groups where VSS/VDS consume separate pNIC(s) on the host. Host transport node consumes separate pNIC(s) for N-VDS installed on the host. N-VDS of the host transport node co-exists with a VSS or VDS, both consuming separate pNICs. Host TEP (Tunnel End Point) and NSX Edge TEP can be in the same or different subnets.
              • NSX Edge VM can be deployed using VLAN-backed logical switches on the N-VDS of the host transport node. Host TEP and NSX Edge TEP must be in different subnets.

              Optionally, you can install multiple NSX Edge appliances/VMs on a single host, and the same management, VLAN, and tunnel endpoint port groups can be used by all installed NSX Edge

               

              NSX Edge Networking Setup

              • 4. Re: NSX-T 3.0 design opinion
                Bayu Wibowo Master
                Community WarriorsUser Moderators

                Hi Pedro,

                 

                You can refer to the NSX-T Design Guide here: VMware® NSX-T Reference Design

                and below VMworld session to get more insights around design

                Next-Generation Reference Design with NSX-T: Part 1 [CNET2061BU]

                Next-Generation Reference Design with NSX-T: Part 2 [CNET2068BU]

                Below in example of 2 pnic with vDS and 2 pnic from the VMworld session

                 

                1 - Do i have any advantage in having only one TZ for VLAN and other TZ for overlay, or have a 3rd TZ (vlan type) for the Edges?

                TZ for VLAN and TZ for Overlay can be on a common single N-VDS

                For Edge's N-VDS, depending on the Edge's uplinks design / North-South topology, if you need a named teaming policy then you would have 3rd probably 4th VLAN TZ for the Edge uplinks with their respective N-VDS.

                So for Edge transport nodes, each will have TZ-Overlay, TZ-Uplink01, TZ-Uplink02. Each TZ-Uplink will have different VLAN ID, uplink profile, N-VDS, and so on

                 

                2 - With this design with a dedicated vDS for the Edges, will i need a different TEP pool for the Edges or i can use the same TEP pool that i use for the hosts?

                You can use the same pool as the Edges are behind vDS and not behind N-VDS.

                If you check VMware HOL NSX-T labs, the TEP for Edge and ESXi are on the same Pool and Edge are behind vDS

                But still better to have them on a separate VLAN & Pool for future proof design e.g. Edge behind N-VDS, vSphere 7 design consideration

                If Edges are behind N-VDS, then you would need a separate TEP Pool + separate VLAN ID for the Edges.

                 

                3 - If i had to use a different TEP pool for the Edges, that vlan have to be trunked on the vDS pnics or only on the N-VDS pnics?

                If Edge N-VDS is connected to VDS and the VLAN of the TEP behind VDS, then you only need to trunk it on VDS pnics.

                Edge TEP to ESXi TEP will go via upstream physical switch so it can communicate between 2 pnics' of VDS to the other 2 pnic's of N-VDS

                 

                4 - What is the advantage of having a vDS for the Edges instead of an N-VDS?

                It depends on design constrains & requirements I would say.

                As you are going to do shared cluster, you may want to think where do you put your ESXi management, vMotion, any other VMkernels (e.g. VSAN)

                Is there any requirement to use vDS for other networks (VSAN, etc)?

                If yes then you will need to keep that vDS regardless and if the Edge will be in the first 2 pnics for North-South and the remaining 2 pnics will be dedicated for East-West then Edge will be behind vDS

                If the first 2 pnics for vDS and used for non-NSX (Management, vMotion, other VMKernels) and other 2 pnics for NSX including Edge, then Edge will be behind N-VDS and require dedicated VLAN + TEP Pool

                Bayu Wibowo | VCIX6-DCV/NV
                Author of VMware NSX Cookbook http://bit.ly/NSXCookbook
                https://github.com/bayupw/PowerNSX-Scripts
                https://nz.linkedin.com/in/bayupw | twitter @bayupw
                • 5. Re: NSX-T 3.0 design opinion
                  Petersaints Novice

                  Hi Bayu,

                   

                  Thanks for the reply.

                   

                  I will have other 2 pNICs for management, vmotion and vSAN.

                   

                  Doubts about your answers:

                  1 - Do i have any advantage in having only one TZ for VLAN and other TZ for overlay, or have a 3rd TZ (vlan type) for the Edges?

                  TZ for VLAN and TZ for Overlay can be on a common single N-VDS

                  For Edge's N-VDS, depending on the Edge's uplinks design / North-South topology, if you need a named teaming policy then you would have 3rd probably 4th VLAN TZ for the Edge uplinks with their respective N-VDS.

                  So for Edge transport nodes, each will have TZ-Overlay, TZ-Uplink01, TZ-Uplink02. Each TZ-Uplink will have different VLAN ID, uplink profile, N-VDS, and so on

                   

                  - TZ-Uplink01, TZ-Uplink02 will be needed to use ECMP?

                   

                   

                  3 - If i had to use a different TEP pool for the Edges, that vlan have to be trunked on the vDS pnics or only on the N-VDS pnics?

                  If Edge N-VDS is connected to VDS and the VLAN of the TEP behind VDS, then you only need to trunk it on VDS pnics.

                  Edge TEP to ESXi TEP will go via upstream physical switch so it can communicate between 2 pnics' of VDS to the other 2 pnic's of N-VDS

                   

                  - If i had Edge N-VDS connected to the vDS, i also need to have edge tep behind the N-VDS, and trunked to the N-VDS pnics, right? Or N-VDS pnics will have only the vlan of the host tep?

                  - Or for better performance should i have only the edge tep vlan behind the vDS and the Edge TEP to ESXi TEP will go via upstream physical switch so it can communicate between 2 pnics' of VDS to the other 2 pnic's of N-VDS?

                   

                  Other doubts:

                  - About the Edge nodes, should they have it's on N-VDS or can i use the N-VDS of the hosts transport nodes?

                  - i'll have 2 x 25Gb and 2 x 10Gb pnics. Should i use the 25Gb for the Edges vDS or for E/W?

                   

                   

                   

                   

                  Thanks.

                   

                  Regards.

                  • 6. Re: NSX-T 3.0 design opinion
                    Petersaints Novice

                    Hello,

                     

                    So in you case if you have a TZ for edges upstream, where do you have connected the tz-vlan for the other vlan backed segments? On different pnics? Other n-vds?

                     

                    Thanks!