VMware Networking Community
jbabcock2017
Contributor
Contributor
Jump to solution

NSX-T Overlay Transport Zones

Hello all, sorry for the newb question, I have deployed two edge nodes in a cluster on one esxi host to start (POC) with that host setup for several overlay transport zones. Why when I try to add those overlay transport zones to my edge nodes do I get this error?

 

[Fabric] Multiple overlay transport zones are not supported on edge node. (Error code: 15517)

 

For those overlay networks to be connect to a T1 then T0 upstream do they not need to be added to the edge nodes?

Confused. 

 

Thanks!

Labels (1)
Reply
0 Kudos
1 Solution

Accepted Solutions
shank89
Expert
Expert
Jump to solution

In order for an edge to provide North South routing for an overlay segment and hosts within that segment, the transport zone the segment is attached to, should be attached to both the host transport node and edge transport node.

The edge Can only belong to one overlay transport zone.

Why are you creating multiple overlay transport zones?

If it's for security you should use dfw, deploy more edges and attach them to the other tz's 

Shashank Mohan

VCIX-NV 2022 | VCP-DCV2019 | CCNP Specialist

https://lab2prod.com.au
LinkedIn https://www.linkedin.com/in/shankmohan/
Twitter @ShankMohan
Author of NSX-T Logical Routing: https://link.springer.com/book/10.1007/978-1-4842-7458-3

View solution in original post

Reply
0 Kudos
9 Replies
shank89
Expert
Expert
Jump to solution

In order for an edge to provide North South routing for an overlay segment and hosts within that segment, the transport zone the segment is attached to, should be attached to both the host transport node and edge transport node.

The edge Can only belong to one overlay transport zone.

Why are you creating multiple overlay transport zones?

If it's for security you should use dfw, deploy more edges and attach them to the other tz's 

Shashank Mohan

VCIX-NV 2022 | VCP-DCV2019 | CCNP Specialist

https://lab2prod.com.au
LinkedIn https://www.linkedin.com/in/shankmohan/
Twitter @ShankMohan
Author of NSX-T Logical Routing: https://link.springer.com/book/10.1007/978-1-4842-7458-3
Reply
0 Kudos
jbabcock2017
Contributor
Contributor
Jump to solution

We are doing it for security zone isolation and segmentation. So is it only one overlay transport zone per cluster or per individual edge node? Also, if you are only able to do one overlay transport zone per node/cluster, are you able to configure multiple ip segments pegged to one overlay transport zone? In other words, if we had 7 separate segments, would we need to deploy 7 different edge gw clusters or 7 different edge nodes?

Thanks!!

Reply
0 Kudos
shank89
Expert
Expert
Jump to solution

Think of a transport zone as the reach of segments.

Ideally you would have a cluster (minimum 2 edge nodes) for HA, and you can only attach edge clusters to T0's and not individual edge nodes.  So it is really 1 overlay transport zone per edge node as you prep the edge node, but you will end up attaching a cluster to a T0.  But you could attach a cluster with a single edge node (not recommended).

For example;

  • Hypervisor cluster test is attached to the test overlay segment.
  • Edge cluster test is attached to the test overlay segment
  • You can have all the segments you want attached to the test overlay segment
  • Each host attached to the test overlay transport zone will see all segments attached to that transport zone
  • The edge cluster will provide north-south routing for the test overlay transport zone, if the edge cluster was not attached to it, you would not be able to have the test edge cluster provide north-south routing for those segments

Let me know if you have any further questions.

 

Shashank Mohan

VCIX-NV 2022 | VCP-DCV2019 | CCNP Specialist

https://lab2prod.com.au
LinkedIn https://www.linkedin.com/in/shankmohan/
Twitter @ShankMohan
Author of NSX-T Logical Routing: https://link.springer.com/book/10.1007/978-1-4842-7458-3
Reply
0 Kudos
grimsrue
Enthusiast
Enthusiast
Jump to solution

The purpose of Transport Zones are for COMPLETE network ISOLATION. Two completely separate domains. One domain will never talk to the other. That means the full data path is completely separate. East/West and North/South, Stateful/Stateless service, segments, etc.

A set of Edge nodes, in an Edge Node Cluster, running two Overlay Transport zones, will mix traffic together north and south traffic from both Transport zones. It would also mix your stateful services together as well. That would defeat the purpose of the total network isolation.

Remember FULL DATA PATH Isolation East/West & North/South.

Each Overlay transport zone has to be on its own set of Edge node, which in turn will also need it own Tier0 which should also have its own BGP peering IPs and AS number as well.
NOTE: If you really wanted to you could configure multiple Tier0's on the same BGP peering connection. It would just need its own peering IPs. I would not suggest doing that though. It will create routing problems.

Remember it is suppose to be FULL DATA PATH Isolation East/West & North South

A transport zone can have many segments connected to it, but a segment cannot connect to more than one Transport Zone.

A single OVERLAY transport zone can only connect to a single cluster of Edge Nodes. If you have 7 Overlay Transport zones then you will need 7 separate Edge Clusters.


An ESXi cluster CAN have multiple Overlay transport zones

I would only use separate Overlay Transport Zones if you are going to do something like run Intranet traffic on one Transport Zone and run Extranet Traffic in a Secondary Transport Zone. You can also use separate transport zones to separate out PROD Network from a TEST Network. Some companies use a separate transport zone per Vendor that they host for Cloud services.

If you were hoping to use transport zone to keep one cluster from seeing the segments of another cluster that would work, but it would be extreme overkill.

engyak
Enthusiast
Enthusiast
Jump to solution

I have to ask, why not run multiple sets of edge transport nodes? In each case, Tier-1 and Tier-0 can become a dedicated N-S firewall (or you can leverage an appliance) and leverage multiple different network paths with what you've already provided.

Is it just because ETNs are a pain to deploy? If so, let's link some Ansible automation(https://github.com/vpackets) and make the transport node deployments easy. They are a bit of work, but having that separation is actually pretty nice if you need it.

You want to deploy Transport Zones as an isolation tool - you do not have to keep a 1:1 radio between segments and transport zones (unless requirements dictate that for, say, DMZs).

Reply
0 Kudos
shank89
Expert
Expert
Jump to solution

I'd be mindful of stating multiple transport zones, while yes it is possible.

 

It's not generally recommended for proper zoning and environment guardrails are much more secure. Not to mention each transport zone requires at least one pnic or two for availability.

 

Whilst yes transport zones provide some kind of separation, they aren't the only way and not necessarily the most efficient use of hardware.

 

 

Shashank Mohan

VCIX-NV 2022 | VCP-DCV2019 | CCNP Specialist

https://lab2prod.com.au
LinkedIn https://www.linkedin.com/in/shankmohan/
Twitter @ShankMohan
Author of NSX-T Logical Routing: https://link.springer.com/book/10.1007/978-1-4842-7458-3
Reply
0 Kudos
engyak
Enthusiast
Enthusiast
Jump to solution

Right - so the primary guidance that out to be used here is that a transport zone defines where a virtual network segment is allowed to propagate, and I was assuming that this was for a direct reason, e.g. multiple infrastructures.

Transport zones are not security zones, they're a tool to define where Jumbo MTU can and cannot flow.

So it is useful to evaluate, knowing this, what the reasoning behind 7 of them are 🙂

jbabcock2017
Contributor
Contributor
Jump to solution

Thanks for the replies all. Things are making more sense. Our concept of the transport zone was thinking it would buy us isolated security zones but it does not sound like that is the case. So if i am understanding correctly, the best approach is to have one transport zone and attach several segments to that zone and use DFW for creating isolation between segments. Having another set of edge nodes with there own T1-T0 would really be for tenant isolation where they can can control there own routing/firewalling. 

Reply
0 Kudos
shank89
Expert
Expert
Jump to solution

It is also possible to create a multi-tenancy approach using a single T0 and additional T1's for each new tenancy.  This is a valid approach and you could use DFW or GW FW rules to box in workload.

The path you take is up to you :).

Please remember to mark the thread as resolved if you are satisfied with any of the responses, and kudo users who have assisted you :).

Shashank Mohan

VCIX-NV 2022 | VCP-DCV2019 | CCNP Specialist

https://lab2prod.com.au
LinkedIn https://www.linkedin.com/in/shankmohan/
Twitter @ShankMohan
Author of NSX-T Logical Routing: https://link.springer.com/book/10.1007/978-1-4842-7458-3