We have been using NSX-v with vSphere 6.5 for several years. We are trying to deploy a new greenfield solution using vCenter 7.0, vSphere 7.0, and NSX-T 3.1 on a single cluster of 3 UCS blade servers. There will be a single vSphere Cluster hosting all VMs - management, normal VM workloads, NSX-T Edges, etc. We plan on using VDS on the ESXi Transport hosts rather than N-VDS. Our UCS blade server profiles are configured with 10 NICs. Eight are currently used for management, vMotion, storage and VLAN based VM traffic. The last two will be dedicated to an additional VDS for use for VM overlay TEP traffic, Edge TEP traffic and Edge VLAN uplinks to the physical routers.
I have reviewed VMware documentation, Udemy courses on NSX-T, the VMware NSX-T 3.0 ICM course and numerous blog postings. I have yet to find clear and concise instructions for how to setup a collapsed cluster using distinct overlay VLANs for ESXi hosts and Edge VMs and with multiple uplinks per Edge.
This blog (https://www.networkbachelor.com/nsx-t-series-part-13-nsx-t-edge-deploy-part-2/) has a diagram of pretty much what we are trying to do but it doesn't go into the details of how exactly to setup the Uplink Profiles, the Transport Zones, the Transport Node Profiles, whether to use Quick Start mode or not, setting up the Edges with the right bits attached to the right things. Its really quite frustrating.
If I try setting up Edge Nodes using Fabric->Nodes,Edge Transport Nodes I end up with Edge VMs with 3 NICs attached (1 for management and two for uplinks -- nothing is attached for TEP use). If I try setting up Edge Nodes using QuickStart->Prepare NSX Transport Nodes, I end up with Edge VMs with four NICs attached (1 for management, 1 for TEP and two for uplinks).
Is there any simple to follow comprehensive documentation anywhere for NSX-T 3.x that covers setting up a collapsed cluster for use with NSX-T, deploying the Edge Nodes, then setting up an initial T-0 and T-1 gateway for connectivity to VMs on NSX-T Segments?
I guess your problem is mainly with uplink profiles. May I know if you are targeting single-Tep or multi-Tep design? Have you created any custom uplink profiles or are we using default uplink profiles? .If possible please share the screenshots of uplink profiles and transport node profiles.
Also please do check https://images.nsx.techzone.vmware.com/sites/default/files/NSX-T%20Reference%20Design%20Guide%203-0.... and search for uplink profiles and teaming policies. We need to ensure we are using the correct uplink profile with teaming policies in the transport node profiles.
I was also confused when using the QuickStart since I don't understand why it didn't follow the recommended configuration for NSX-T 2.5 and later. I will look into that more later.
For your case I would try again to follow "Fabric-> Nodes, Edge Transport Nodes I end up with Edge VMs with 3 NICs attached (1 for management and two for uplinks". Note that the TEPs will use both Uplinks, so you don't need a separate vNIC for that. The fourth vNIC should be disconnected unless you are going to use bridging.
Check this out for a step-by-step guide:
Generate and Register the NSX-T Management SSL Certificate and Private Key Create the NSX-T Management cluster by installing 3 NSX Manager the NSX-T Management VIP address so that the the cluster certificate.