Highlighted
Enthusiast
Enthusiast

NSX-T 3.0 design opinion

Hello all,

I will need to deploy NSX-T 3.0 on a shared cluster (vSphere 6.7) with 4 pnics to dedicate to NSX-T and i'm thinking on the following design:

- 2 pnic assign to a vDS for the Edges

- 2 pnics assign to a N-VDS

My doubts are:

1 - Do i have any advantage in having only one TZ for VLAN and other TZ for overlay, or have a 3rd TZ (vlan type) for the Edges?

2 - With this design with a dedicated vDS for the Edges, will i need a different TEP pool for the Edges or i can use the same TEP pool that i use for the hosts?

3 - If i had to use a different TEP pool for the Edges, that vlan have to be trunked on the vDS pnics or only on the N-VDS pnics?

4 - What is the advantage of having a vDS for the Edges instead of an N-VDS?

Thanks.

Kind regards,

0 Kudos
6 Replies
Highlighted
Contributor
Contributor

Is this a production deployment or for lab purposes? Your design choices are influenced by the underlying network. For example, we have dedicated vlan transport zones for our edge nodes that are just used for the connections to the upstream network. The edge nodes don't need to connect to our other vlan-backed segments that are used by our workload VMs that haven't been moved to an overlay network.

0 Kudos
Highlighted
Enthusiast
Enthusiast

Hi,

It's for production. In my case, i was thinking in use the edge's vlan's just to N/S.

0 Kudos
Highlighted
VMware Employee
VMware Employee

Any specific reason why you are opting for NVDS based approach ?  I would highly recommend to go with VDS 7.0 ( if it is feasible to upgrade vsphere to 7.0) based integration since it is fully supported design.

1 - Do i have any advantage in having only one TZ for VLAN and other TZ for overlay, or have a 3rd TZ (vlan type) for the Edges?

VLAN based TZ is for what purpose ? Is it just for connecting VLAN backed networks ?

2 - With this design with a dedicated vDS for the Edges, will i need a different TEP pool for the Edges or i can use the same TEP pool that i use for the hosts?

3 - If i had to use a different TEP pool for the Edges, that vlan have to be trunked on the vDS pnics or only on the N-VDS pnics?

4 - What is the advantage of having a vDS for the Edges instead of an N-VDS?

NSX Edge VM can be installed on an NSX-T Data Center prepared host and configured as a transport node. There are two types of deployment:

  • NSX Edge VM can be deployed using VSS/VDS port groups where VSS/VDS consume separate pNIC(s) on the host. Host transport node consumes separate pNIC(s) for N-VDS installed on the host. N-VDS of the host transport node co-exists with a VSS or VDS, both consuming separate pNICs. Host TEP (Tunnel End Point) and NSX Edge TEP can be in the same or different subnets.
  • NSX Edge VM can be deployed using VLAN-backed logical switches on the N-VDS of the host transport node. Host TEP and NSX Edge TEP must be in different subnets.

Optionally, you can install multiple NSX Edge appliances/VMs on a single host, and the same management, VLAN, and tunnel endpoint port groups can be used by all installed NSX Edge

NSX Edge Networking Setup

Cheers,
Sree | CKA|VCAP-NSX-T| VCIX-3X| VCAP-3X| VExpert 4x
0 Kudos
Highlighted
User Moderator
User Moderator

Hi Pedro,

You can refer to the NSX-T Design Guide here: VMware® NSX-T Reference Design

and below VMworld session to get more insights around design

Next-Generation Reference Design with NSX-T: Part 1 [CNET2061BU]

Next-Generation Reference Design with NSX-T: Part 2 [CNET2068BU]

Below in example of 2 pnic with vDS and 2 pnic from the VMworld session

pastedImage_4.png

1 - Do i have any advantage in having only one TZ for VLAN and other TZ for overlay, or have a 3rd TZ (vlan type) for the Edges?

TZ for VLAN and TZ for Overlay can be on a common single N-VDS

For Edge's N-VDS, depending on the Edge's uplinks design / North-South topology, if you need a named teaming policy then you would have 3rd probably 4th VLAN TZ for the Edge uplinks with their respective N-VDS.

So for Edge transport nodes, each will have TZ-Overlay, TZ-Uplink01, TZ-Uplink02. Each TZ-Uplink will have different VLAN ID, uplink profile, N-VDS, and so on

2 - With this design with a dedicated vDS for the Edges, will i need a different TEP pool for the Edges or i can use the same TEP pool that i use for the hosts?

You can use the same pool as the Edges are behind vDS and not behind N-VDS.

If you check VMware HOL NSX-T labs, the TEP for Edge and ESXi are on the same Pool and Edge are behind vDS

But still better to have them on a separate VLAN & Pool for future proof design e.g. Edge behind N-VDS, vSphere 7 design consideration

If Edges are behind N-VDS, then you would need a separate TEP Pool + separate VLAN ID for the Edges.

pastedImage_0.png

3 - If i had to use a different TEP pool for the Edges, that vlan have to be trunked on the vDS pnics or only on the N-VDS pnics?

If Edge N-VDS is connected to VDS and the VLAN of the TEP behind VDS, then you only need to trunk it on VDS pnics.

Edge TEP to ESXi TEP will go via upstream physical switch so it can communicate between 2 pnics' of VDS to the other 2 pnic's of N-VDS

4 - What is the advantage of having a vDS for the Edges instead of an N-VDS?

It depends on design constrains & requirements I would say.

As you are going to do shared cluster, you may want to think where do you put your ESXi management, vMotion, any other VMkernels (e.g. VSAN)

Is there any requirement to use vDS for other networks (VSAN, etc)?

If yes then you will need to keep that vDS regardless and if the Edge will be in the first 2 pnics for North-South and the remaining 2 pnics will be dedicated for East-West then Edge will be behind vDS

If the first 2 pnics for vDS and used for non-NSX (Management, vMotion, other VMKernels) and other 2 pnics for NSX including Edge, then Edge will be behind N-VDS and require dedicated VLAN + TEP Pool

Bayu Wibowo | VCIX6-DCV/NV Author of VMware NSX Cookbook http://bit.ly/NSXCookbook https://github.com/bayupw/PowerNSX-Scripts https://nz.linkedin.com/in/bayupw | twitter @bayupw
0 Kudos
Highlighted
Enthusiast
Enthusiast

Hi Bayu,

Thanks for the reply.

I will have other 2 pNICs for management, vmotion and vSAN.

Doubts about your answers:

1 - Do i have any advantage in having only one TZ for VLAN and other TZ for overlay, or have a 3rd TZ (vlan type) for the Edges?

TZ for VLAN and TZ for Overlay can be on a common single N-VDS

For Edge's N-VDS, depending on the Edge's uplinks design / North-South topology, if you need a named teaming policy then you would have 3rd probably 4th VLAN TZ for the Edge uplinks with their respective N-VDS.

So for Edge transport nodes, each will have TZ-Overlay, TZ-Uplink01, TZ-Uplink02. Each TZ-Uplink will have different VLAN ID, uplink profile, N-VDS, and so on

- TZ-Uplink01, TZ-Uplink02 will be needed to use ECMP?

3 - If i had to use a different TEP pool for the Edges, that vlan have to be trunked on the vDS pnics or only on the N-VDS pnics?

If Edge N-VDS is connected to VDS and the VLAN of the TEP behind VDS, then you only need to trunk it on VDS pnics.

Edge TEP to ESXi TEP will go via upstream physical switch so it can communicate between 2 pnics' of VDS to the other 2 pnic's of N-VDS

- If i had Edge N-VDS connected to the vDS, i also need to have edge tep behind the N-VDS, and trunked to the N-VDS pnics, right? Or N-VDS pnics will have only the vlan of the host tep?

- Or for better performance should i have only the edge tep vlan behind the vDS and the Edge TEP to ESXi TEP will go via upstream physical switch so it can communicate between 2 pnics' of VDS to the other 2 pnic's of N-VDS?

Other doubts:

- About the Edge nodes, should they have it's on N-VDS or can i use the N-VDS of the hosts transport nodes?

- i'll have 2 x 25Gb and 2 x 10Gb pnics. Should i use the 25Gb for the Edges vDS or for E/W?

Thanks.

Regards.

0 Kudos
Highlighted
Enthusiast
Enthusiast

Hello,

So in you case if you have a TZ for edges upstream, where do you have connected the tz-vlan for the other vlan backed segments? On different pnics? Other n-vds?

Thanks!

0 Kudos