NSX-T TEP pools best practices

[Petersaints]

Hello all,

I read on some blogs that when we have to pass host tep and edge tep on the same pnic (compute and management on the same cluster), we need two different tep pools. I also read that one alternative is to have de edges connected to a vDS (e.g management, that uses other pnics), and the hosts tep connected to a n-vds, that will use the other 2 pnic, right?

In your opinion what is the correct design, the simple one? One vDS + N-VDS, or two tep pools? What are the advantages of the two scenarios?

Many thanks.

Pedro

[daphnissov]

The question isn't really about TEP pools moreso than it's about separate or shared pNICs for the N-VDS. When you have everything running over an N-VDS, you require separate networks for host transport nodes and edge transport nodes. Those separate networks mean separate TEP pools. By contrast, if you have you host transport nodes on dedicated pNICs using an N-VDS and you edge nodes are virtual connected to a vDS, you can use the same network for the TEPs (traffic still leaves the host), therefore you can have just a single TEP pool. This changes when it comes to vSphere 7 because the vDS and N-VDS are collapsed into one entity, so therefore you need those separate networks.

[HassanAlKak88]

Hello,

For Edges deployed on Host NVDS, it is mandatory to have separate VLANs for Host TEP and Edge TEP network. check the following blog:  https://vxplanet.com/2019/10/22/nsx-t-single-nvds-multi-tep-edge-vm-deployment-configuration-on-host...

[Petersaints]

Hi HassanAlKak88,

Thanks for the reply. I'm on my first steps on NSX-T and i have some doubts. Following the blog suggested, in this scenario that we have only two pnics and one n-dvs, can you explain me, please:

1 - TZ-Overlay will be selected on Host transport nodes and edge transport nodes configuration? (vlan 80 edge overlay tranport and vlan 40 host overlay transport)

2 - Why on the edges deployment, the 4th nic will be removed? Shouldn't be for overlay traffic?

Many thanks.

Pedro

[HassanAlKak88]

Hello,

Look dear, with NSX-T especially with the old versions there were many supported deployments (Single N-VDS/Multi-TEP, Multiple N-VDS/single TEP, shared cluster, management & edge cluster.....etc).

The shared topology is Single NVDS with vDS (each with two separate pNics) on a shared cluster. For the Overlay TZ yes you will use it on the two profiles/configuration (host and edge) and you will create two different pools for TEP one for host and one for edge (sure with two different VLANs and ensure the L3 connectivity between them).

And with Single NVDS for edges, you will use the management eth0 adapter and two FP ports with Active/Active LB teaming policy for TEP and Uplinks. The 4th one not used and no need to use with this topology.

[Petersaints]

Hello,

Now i'm confused!

In my scenario i only have 3 pnics available. One is connected to a standard switch where i have the management network of the hosts, vcenter, nsx manager, and where i will connect only the management interface or the edges. The other 2 pnics i will use for the N-VDS. So my scenario will be  "NSX-T single N-VDS Multi-TEP", right?

Thanks.

[HassanAlKak88]

Look when we were talking about single N-VDS for the edge we talking about the edge itself, so all edge's up-links (management and FP) are using port groups on a vDS. And all hosts are under  this vDS. so in this config we cover the up-links of edges, TEP and managements.

For host, the overlay (TEP) use different two uplinks for NVDS with the TEP of hosts.

But in your case, you have 3 pNics on server one with standard swith and the others are for NVDS. so in this case you have two option:

you can use the above but it is risky and not highly available for edge uplinks and TEPs using the standard port groups

or you have to go with a different topology, everything under NVDS. in this case you will use the VLAN tagged logical segment with VLAN transport zone. Check the following: Fully Collapsed vSphere Cluster NSX-T Deployment

[Petersaints]

Hello again,

So let me see if i understood. The perfect scenario for a shared compute and edge deployment is:

- one nvds for host (2 pnic) +  one nvds for edges (2 pnic)?

Thanks.

[daphnissov]

There is no "perfect scenario" for these things. It depends on what you have in your environment. It works either way just fine: 2 pNICs on vDS + 2 pNICs on N-VDS or all pNICs on N-VDS. The question begins with "how many pNICs can I give to NSX-T?" and then go from there.

[HassanAlKak88]

Yeh, it is perfect if you have 4 pNics.

All these options should be discussed during the design phase of NSX-T before the deployment and sure the decision related to the existing HW with the existing resources.

For more information don't hesitate to contact me,

Please consider marking this answer "correct" or "helpful" if you think your question has been answered correctly.

[LeitaoL]

Hi Peter,

In short, answering directly to your initial question, you can have a single VLAN for Host and Edge TEP's. But, it is a common best practice to separate TEP traffic between Hosts and Edges.

Take a look at this scenario to understand the importance of separate TEP Networks for Host and Edge: https://www.spillthensxt.com/nsx-t-tep-ip-addressing/

I had this doubt myself, coming from NSX-V to NSX-T, and will continue to have many others.

Since NSX-T 3.0 and vSphere 7, you can have a single VDS with all the traffic that you would normally have in your datacenter..

- Logical VLANs from the Core Switching Infrastructure

- NSX Networking Segments

- L2 Bridged-Backed Segments

All you have to do is, convert your existing Standard Switch or Distributed Switch, during the Host preparation.

With this possibility, you can easily separate TEP traffic 😉

Cheers

[Petersaints]

Hello again,

Other silly question:

In a scenario that i'm using 4 pnics, 2 of them on a dvs where i have esxi management, vmotion,vsan, nsx manager management network, edges management network, the other 2 pnics i will use for nsx. On nsx, i will have a cluster of edges (two edges). My question is: i will have only one NVDS to connect the 2 egdes, or, i must have 2 nvds, one for each edge?

Thanks.

Pedro

[HassanAlKak88]

Hello,

Noting that you used the first 2*pNics for Mgmt, VSAN, vMotion, edge uplinks, and edge TEP,.....etc (so the NVDS for edges will connect to this vDS at the end)

The other 2*pNics will be used for host TEP which means all VM traffic.

[Petersaints]

Hi,

So, when a vm need to contact the physical world, the traffic will go by the vDS?

[HassanAlKak88]

Correct,

[Petersaints]

Hi again,

So if i had only 4 pnics and want to separate all the VM traffic, from the esxi management, vmotion, vsan... i should:

1 - create a vds for esxi management, vmotion, vsan (2 pnic)

2 - create a single nvds (2 pnic)

    - two transport zones (overlay + vlan)

    - Two uplink profiles (compute + edges)

3 - create segments

    - if my edges management network is the same of the esxi hosts, i must create that segment, right?

4 - Deploy the edges

Thanks.

[HassanAlKak88]

Hi,

You have two types of VM traffics East/West and North/South.

East/West traffic with NSX-T, this will be handled through the Overlay GENEVE tunnel using the two physical separated uplinks for NVDS.

For North/South traffic, the communication with the physical world you have to use two different uplinks connected to the next hop (L3 device) to route you between the NSX domain and other networks.

It is recommended to separate these two types from uplinks sides as per VMware.

[Petersaints]

Hi,

So your suggestion is two pnics for N/S and other two pnics for E/W?

[HassanAlKak88]

Yeh and the pNics for N/S traffic will be shared with vShpere management, vMotion, VSAN, VLAN tagged traffic,....etc  using the vDS and port groups

And all E/W traffic will be using the pNics and NVDS.