All TKB Articles in VMware NSX

Modern apps need to run in multi-cluster, multi-cloud environments across a mix of traditional and microservices architectures. In this context, enterprise platform, infrastructure, and operations teams are presented with unique challenges in securely connecting and managing modern workloads, in delivering scalable services, or bridging between traditional VM workloads and containers, and supporting production operations for modern apps.   VMware recently introduced the “VMware Modern Apps Connectivity solution”, which brings together the advanced capabilities of Tanzu Service Mesh (TSM) and VMware NSX Advanced Load Balancer ALB (formerly Avi Networks) address today’s unique enterprise challenges.      

We are excited to announce the general availability of VMware NSX-T™ 3.0, a major release of our full stack Layer 2 to Layer 7 networking platform that offers virtual networking, security, load balancing, visibility, and analytics in a single platform. NSX-T 3.0 includes key innovations across cloud-scale networking, security, containers, and operations that help enterprises achieve one-click public cloud experience wherever their workloads are deployed. As enterprises adopt cloud, containers, and new applications, IT teams are managing more heterogenous and distributed environments that need to be better secured, automated, and monitored. The need to run and manage workloads on all types of infrastructure, VMs, containers, bare metal across both private and public clouds, is greater than ever. Enterprises need end-to-end software-defined solutions to fully automate, connect, and protect all their workloads.  As a key component of VMware Virtual Cloud Network, VMware NSX-T 3.0 includes groundbreaking innovations that make it easier to replace legacy appliances that congest data center traffic, achieve stronger security posture, and run virtual and containerized workloads anywhere. NSX-T 3.0 also introduces global policy consistency, AWS and Azure gov cloud support, VMware NSX® Intelligence enhancements, Layer 3 EVPN, and powerful networking and security services for vSphere with Kubernetes, superseding features in NSX-V. In addition, NSX-T 3.0 integrated with enhancements in VMware vRealize™ Network Insight 5.2 to deliver comprehensive end-to-end network visibility and flow-based application discovery. Cloud-scale Network Agility Scaling up and managing a cloud environment – whether public or private – requires simplified network configuration and management, visibility and control, and the ability to rapidly add new capabilities into the existing environment.  NSX Federation – NSX Federation in NSX-T 3.0 helps deliver a cloud-like operating model by simplifying the consumption of networking and security constructs. It introduces the NSX Global Manager, a centralized console for managing the network as a single entity while keeping configuration and operational state synchronized across multiple locations.        Security policies attach and move with the workload, ensuring that policy compliance is maintained during workload failover or migration between locations. Follow us on twitter @vmwarensx for a detailed blog on NSX Federation in a few weeks.  Support for AWS GovCloud and Azure Government – NSX-T 3.0 extends support for public clouds with VMware NSX™ Cloud support for AWS GovCloud and Azure Government. This provides isolated public cloud environments for U.S. government agencies and customers to move sensitive workloads into the cloud and assist with regulatory and compliance requirements. NSX customers will benefit from the extended visibility, consistent networking and security policies, precise control over cloud networking, and end-to-end operational control across clouds.  Enhanced Multi-tenancy with VRF Lite and Layer 3 BGP EVPN – VRF Lite greatly reduces the networking infrastructure footprint by introducing complete data plane tenant isolation with separate routing table, NAT, and firewall within each VRF on NSX Edge. NSX Edge also implements Layer3 EVPN to seamlessly connect telco VNFs to the overlay network.  The Edge implements standards based BGP control plane to advertise IP Prefixes, running eBGP sessions to the VNF and MP-BGP sessions with the PE/DCGW(s).  Dynamic Network Service Chaining – NSX service insertion is further enhanced with support for dynamic service chaining for traffic from and to VMs, containers, and bare metal workloads.  The Edge Node dynamically classifies incoming network traffic and applies a set of network services to achieve app-aware security and monitoring.  Best-in-class Intrinsic Security With NSX-T 3.0, the Service-defined Firewall in the NSX platform has been enhanced with the addition of a distributed IDS/IPS  solution to protect east-west traffic in the data center. NSX-T 3.0 is a step further towards our goal of extending the NSX intrinsic security approach from every workload to data center, multi-cloud, and edge.  NSX Distributed IDS/IPS – At VMworld Europe last year, we announced the VMware Distributed IDS/IPS solution for our advanced Layer 7 internal firewall. NSX Distributed IDS/IPS is an advanced threat detection engine purpose-built to detect lateral threat movement on east-west traffic across multi-cloud environments.  It eliminates security blind-spots and helps meet compliance needs.     Unlike traditional architectures that hairpin traffic to discrete appliances, NSX Distributed IDS/IPS distributes the analysis out to every workload and curates the signatures evaluated by each engine based on precise knowledge of running applications. This elastic throughput scales with workloads while improving utilization of existing compute capacity, simplifies the network design and operational model, and radically reduces the rate of false positives. This approach enables security teams to replace discrete appliances, and helps achieve regulatory compliance and create virtual security zones without physical separation of infrastructure.  L7 Edge Firewall Enhancements – The Layer 7 Edge Firewall is further enhanced in NSX-T 3.0 with the implementation of URL Analysis for URL Classification and Reputation. The Edge Firewall detects access from outside the datacenter for granular detection and categorization of in-bound and outbound URLs. DFW for Windows 2016 workloads – In addition to existing support for Linux, NSX-T 3.0 adds NSX Distributed Firewall (DFW) support for Windows 2016 based physical workloads.  Time-based rules and Configuration wizard – Firewall rules can be enforced based on a pre-scheduled timeline defined by the administrator. NSX-T 3.0 also simplifies the implementation of VLAN backed micro-segmentation using a new configuration wizard.  Full-stack Networking and Security for Modern Apps Networking for vSphere with Kubernetes – NSX-T is designed-in from the ground up as the default pod networking solution for vSphere with Kubernetes.  NSX provides a rich set of networking capabilities for vSphere with Kubernetes, including distributed switching and routing, firewalling, load balancing, NAT, IPAM, and more.     Vinay Reddy describes how NSX-T, designed into vSphere with Kubernetes as the default networking solution, addresses common challenges associated with container networking and security.     Prescriptive networking for vSphere Namespace isolation – NSX-T 3.0 delivers a prescriptive network design to greatly simplify the implementation of vSphere Namespaces. It automatically implements the logical segments, distributed routing and firewalling, and IPAM services required for Namespace isolation in the vSphere Supervisor Cluster.  Any workloads created in a Namespace automatically inherit the security policy applied to that Namespace, allowing developers to self-service resources into that Namespace.  Integration with Cluster API in VMware Tanzu Kubernetes Grid Service – NSX-T integrates with VMware Tanzu Kubernetes Grid Service to allow developers to deploy Tanzu Kubernetes Grid clusters.  NSX-T greatly simplifies the necessary networking infrastructure, including the creation of logical segments, Tier-1 Gateway, and load balancers, needed for Tanzu Kubernetes Grid clusters. Operational Simplicity and Automation  Converged vSphere® Distributed Switch™ – With NSX-T 3.0, admins can now deploy NSX-T directly on VMware vSphere Distributed Switch 7.0. This greatly simplifies NSX-T deployment in vSphere environments with no changes required to the existing vSphere Distributed Switch and no VM traffic disruption.     Policy Enhancements with Terraform Provider & Ansible Module – NSX-T 3.0 extends the use of Terraform Provider and Ansible Modules, two of the most widely used automation tools for config generation and deployment, beyond NSX-T installation use cases with support for the NSX-T Policy API.  It now supports additional topology deployment workflows for security, logical gateway and segments, and network overlays and VLAN segments. Lifecycle management has become easier with the Ansible Module. NSX-T component upgrade of NSX Managers, Transport Nodes, Edge Nodes can be automated with the Ansible Modules.  Simplified Integration with vRealize Network Insight 5.2 – Tight integration with vRealize Network Insight 5.2 delivers comprehensive end-to-end network visibility. Support for vRealize Operations alerts enables precise troubleshooting in NSX-T environments from vRealize Network Insight dashboard. vRealize Network Insight 5.2 also implements flow-based application discovery across VMware platforms for application categorization by tier.  OpenStack Neutron Enhancements – The OpenStack Neutron plugin for NSX-T has been enhanced to abstract multiple NSX-T end points and operators can now configure additional IPv6 features (including DHCPv6, IPv6 LB, and NAT64) using the NSX-T policy plugin.  Summary  The NSX-T 3.0 release expands the breadth and depth of NSX-T use cases across cloud-scale networking, distributed IDS for advanced threat protection, and modern container-based applications. We remain committed to helping our customers radically simplify their network, achieve consistent policies across locations and transform their operations in the data center and cloud with full-stack automation across switching, routing, security, load balancing, and other layer 7 network services.  Follow us on Twitter @vmwarensx for updates and a series of deep-dive blogs on the key capabilities delivered in NSX-T 3.0.  NSX-T Resources  VMware NSX-T 3.0 Resources  More on NSX-T 3.0 Read the Press Release  Download NSX-T 3.0  Get started with a Beginner or Advanced NSX Hands-On-Lab (HOL)  Take a Virtual Cloud Network Assessment VMware NSX Product Page VMware NSX YouTube Channel, including 45+ Light Board videos!  VMware vRealize Network Insight (vRNI) 5.2 Resources  vRNI 5.2 Release Blog  vRNI Product Page vRNI Lighting Lab

This presentation teaches you how ECMP can be leveraged within NSX-T.   There are multiple layers of ECMP within NSX-T:  ECMP between the DR to SR  ECMP between the SR and the physical networking fabric.  NSX-T offers an optimal way to load balance the networking traffic within the Data Center.   We will also present how easy it is to enable ECMP whether you use BGP, OSPF or static routes.  Do not hesitate to comment or to ask questions. Video: https://www.youtube.com/watch?v=XdSs-S2a5fc April 2021: v1.2  June 2021: v1.3 - Added ECMP DR to SR for Multi-Tier routing architecture

  This is the VMware® NSX-T 3.1 Security Configuration Guide.This guide provides prescriptive guidance for customers on how to deploy and operate VMware® NSX-T in a secure manner. Guide is provided in an easy to consume spreadsheet format, with rich metadata (i.e. similar to existing NSX for vSphere & VMware vSphere Security Configuration Guides) to allow for guideline classification and risk assessment. Feedback and Comments to the Authors and the NSX Solution Team can be posted as comments to this community Post (Note: users must login on vmware communities before posting a comment). Other related NSX Security Guide can be found @ https://communities.vmware.com/docs/DOC-37726 --The VMware NSX PM/TPM Team

  This presentation provides all the supported capabilities regarding OSPFv2 support in NSX-T 3.1.1:     Active/Active and Active/Standby Topology Point to Point and Broadcast Network Standard / Backbone / NSSA Areas Area Definition (Authentication) CLI Outputs Route redistribution Summarization UI Configuration Feel free to ask any questions below ! Thanks

This document is a complete reference on the VMware NSX Advanced Load Balancer (by Avi Networks). The PowerPoint document is organized as slides that cover the following topics in detail: 1 Architecture/Infrastructure 2 Monitors 3 Server Pools 4 Layer 4 VIP 5 Layer 7 HTTP VIP 6 Layer 7 HTTPS VIP 7 Profiles and Policies 8 Manipulating Traffic Flows 9 Application Troubleshooting 10 Events and Alerts      

Authors: VMware NSX Technical Product Management Team This is the NSX-T Reference Design Page. The latest doc updates are aligned to NSX version 4.1. Highlights include: Technology overview chapters: TEP HA (Ch3) VPC/Projects (Ch2) A/A Stateful Gateways (Ch4) Bare Metal edge hardware recommendation (Ch8) DPUs (Ch9) Update VRF route leaking (Ch4) Design Chapter (Ch7): Projects 1 VC to many NSX MTU recommendation, Gateway vs. global MTU   Readers are encouraged to send feedback to NSXDesignFeedback_at_groups_vmware_com (replace at and underscores) We will continue updating this document, so please re-download this document. --The VMware NSX Product Management

NSX-T Security Reference Guide -  This talks about NSX Service-defined Firewall capabilities, different use cases, architecture, consumption model and the best practices around the security design.   1.3 version mainly has following updates along with minor update to all section: * Chapter -1: NSX Service-defined firewall value prop/positioning. * Chapter -2: NSX Use cases – What/why/how and NSX deployment Options. * Chapter -5: Best practices around Groups/Tags/Policy 
Readers are encouraged to send a feedback to NSXDesignFeedback_at_groups*vmware*com  (replace _at_  -> @ and * -> .) 
We will continually updating this document so please re-download this document. 
--The VMware NSX Product Management

NSX-T 3.0 Operation Guide  Authors: VMware NSX Technical Product Management Team   This is the NSX-T Operation Guide based on NSX-T release 3.0. It is the foundational overhaul to NSX-T day 2 operation guidance and some of best practices.  It covers: NSX-T Monitoring Tools NSX-T Troubleshooting Tools NSX-T Operation best practices Readers are encouraged to send a feedback to NSXDesignFeedback_at_groups_vmware_com  (replace at and underscores) We will continually updating this document so please re-download this document.   --The VMware NSX Product Management

This deck offers a nice presentation of what is NSX Federation and how it works. A very similar deck was used at VMworld 2020 session VCNC1178D here and watching is a nice option as it gives "voice over" the deck. Federation demos are also available here.   Note1: This ToI may be updated in the future so always check you have the latest version.     . NSX 4.0-4.1 Federation 101 ToI version is 1.0 done on 10/30/2023.     . NSX-T 3.2 Federation 101 ToI version is 1.1 done on 08/26/2022.   Note2: For deeper information, we also offer the "NSX Federation Multi-Location Design Guide (Federation + Multisite)" here.

NSX-T offers two technical solutions for Multi-Locations On-Prem Data Centers: NSX-T Federation NSX-T Multisite This NSX-T Multi-Location Design Guide offers guidance and best practices for Network & Security services in your On-Prem locations. FYI there is also some other nice documents on this use case: NSX-T Federation Presentation (ppt deck here with a link to demos) NSX-T Multisite Presentation (ppt deck here with embedded demos)   Note: This document may be updated in the future so always check you have the latest version. The Design Guide version for NSX-T 4.1 release is 1.4 done on 08/22/2023. The Design Guide version for NSX-T 4.0 release is 1.10 done on 08/22/2023. The Design Guide version for NSX-T 3.2 release is 1.19 done on 08/22/2023. The Design Guide version for NSX-T 3.1 release is 1.31 done on 06/21/2023.  

Three Federation demos are proposed here:  1. Federation Network & Security Services demo: "Federation-Demo1-Network&Security.mp4" 2. Federation Disaster Recovery(Network/Security & Compute) with Stretched Networks + SRM: "Federation-Demo2-DR.mp4" 3. Federation Disaster Recovery(Network/Security & Compute) with GSLB: "Federation-Demo3-DR_GSLB.mp4"   Enjoy the demos. Dimitri   Note1: For information on NSX-T Federation we offer the "NSX-T Federation Presentation" here. Note2: For deeper information, we also offer the "NSX-T Federation Multi-Location Design Guide (Federation + Multisite)" here.

This document attempts to describe the use cases, the configuration, the redundancy model and the design scenarios related to the NSX-T edge bridge. Even if we're talking about a 30 page white paper on a very specific topic, I can already see some few areas missing coverage. However, I think this is already useful enough to be shared to the world. Let me know you feedback on clarity. I'm more interested in making sure this is easy to understand than expanding the scope of this piece at that stage. Thanks and regards, Francois Tallet @ vmware Jan 15th 2021: updated for 3.1 (CLI changes)  

Proxy ARP on the NSX-T edge node is a feature supported since NSX-T 2.4. Proxy ARP is automatically enabled when a NAT rule or a load balancer VIP uses an IP address from the subnet of the Tier-0 gateway uplink. Proxy ARP can be considered in environments where IP subnets are limited and where it is problematic to use new subnets easily and rapidly (either by using static routes or BGP). For production environment, VMware recommends implementing proper routing between a physical fabric and the NSX-T Tier-0 by using either static routes or Border Gateway Protocol with BFD. If proper routing is used between the Tier-0 gateway and the physical fabric, BFD with its sub-second timers will converge faster. The following table summarizes the design option and support for the Proxy ARP feature on the NSX-T Edge: NSX-T Proxy ARP Support Tier-0 High Availability Mode Support Active / Standby Supported Active / Active Not supported For more details regarding Proxy-ARP on NSX-T please refer to the document below This document was generated from the following discussion: The specified discussion was not found.

This document is an informal document that walks through the step-by-step deployment and configuration workflow for NSX-T Edge Single N-VDS Multi-TEP design.  This document uses NSX-T 3.0 UI to show the workflow, which is broken down into following 3 sub-workflows: 1- Deploy and configure the Edge node (VM & BM) with Single-NVDS Multi-TEP. 2- Preparing NSX-T for Layer 2 External (North-South) connectivity. 3- Preparing NSX-T for Layer 3 External (North-South) connectivity. Hope this helps. --The VMware NSX Product Management

  This is the VMware® NSX-T 3.0 Security Configuration Guide.This guide provides prescriptive guidance for customers on how to deploy and operate VMware® NSX-T in a secure manner.   Guide is provided in an easy to consume spreadsheet format, with rich metadata (i.e. similar to existing NSX for vSphere & VMware vSphere Security Configuration Guides) to allow for guideline classification and risk assessment. Feedback and Comments to the Authors and the NSX Solution Team can be posted as comments to this community Post (Note: users must login on vmware communities before posting a comment). Other related NSX Security Guide can be found @ https://communities.vmware.com/docs/DOC-37726 --The VMware NSX PM/TPM Team

This procedure is based on KB 57844 When you have overlapping segment ID Pool range in a specific environment (one vCenter server) with another environment (second vCenter server) this is the full process how to migrate the current working objects (VMs, NSX Edges, Logical Routers, Logical Switches) to a new Segment ID Pool: I. Prerequisite 0. Put monitoring suppression for vCD Cells, vCenter server and NSX Manager 1. Upgrade all components to 6.4.4: NSX Manager, NSX Controllers, host agents, Edge Gateways 2. Stop the backups (if they are using the vCenter server API) 3. Setup Postman 3.1. Download and start Postman 3.2. Create a request 3.3. Headers > Key "Content Type" > Value "application/xml" 3.4. Authorization > Basic Auth > username "admin" > password 3.5. File > Settings > Turn off "SSL Certificate Verification" 4. Stop the vCenter server operations from vCD: login vCD (https://yourowncloud.com): Manage & Monitor > vCenters > Right click on the vCenter server > Disable 5. Change cluster DRS configuration from Fully Automated to Manual 6. Gather information (with PowerShell) in CSV file, about all NSX objects which will be migrated: Logical Switches, Logical Routers, Edges, VMs, etc. (script bellow just collect data): [CmdletBinding(PositionalBinding=$false)] Param ( [parameter(Position= 0, Mandatory = $false)] [string]$VIServer = "YOURCLOUDVCR01.local", [string]$PathExportNsxLogicalSwitch     = "C:\Support\Scripts\NSX\NSXReport.csv", [string]$PathExportNsxLogicalRouter     = "C:\Support\Scripts\NSX\NSXLogicalRouter.csv" ) begin {     If ( ! (Get-module PowerNSX )) {     Import-Module PowerNSX     } # connecting to the NSX server $connection = Connect-NSXServer -vCenterServer $VIServer $defaultNsxConnection = $connection $defaultViServer = $connection.viConnection } process { # Getting NSX Edge information $getEdge = get-nsxedge |Get-NsxEdgeInterface $edge = $getEdge | select name,edgeId,portgroupName $edgeEdgeSub = $getEdge | Get-NsxEdgeSubInterface # Getting NSX Logical Router information $getNsxLogicalRouter = Get-NsxLogicalRouter | Get-NsxLogicalRouterInterface | select connectedToId,logicalRouterId,connectedToName,type     $output =    foreach ( $ls in Get-NsxLogicalSwitch ) {         $pg = $ls | Get-NsxBackingPortGroup         foreach ( $portgroup in $pg) {             $vm = $portgroup| Get-VM             foreach ( $virtualmachine in $vm) {                     $vlookup = $edge | where {$_.portgroupName -like $ls.name}                     $vlookupEdgeSub = $edgeEdgeSub | where {$_.logicalSwitchName -like $ls.name}                     $VMdetails = (get-vm $virtualmachine.name | Get-NetworkAdapter | where {$_.NetworkName -like $portgroup.name})                  [pscustomobject]@{                     "vCenter" = $defaultViServer.name                     "NSX" = $defaultNsxConnection.server                     "LS_ObjectID" = $ls.objectId                     "LS_Name" = $ls.name                     "LS_vdnId" = $ls.vdnId                     "EdgeID" = $vlookup.edgeId                     "EdgeVNIC" = $vlookup.name                     "EdgeTrunk_LS_ID" = $vlookupEdgeSub.logicalSwitchId                     "EdgeTrunk_LS_Name" = $vlookupEdgeSub.logicalSwitchName                     "EdgeTrunk_LS_isConnected" = $vlookupEdgeSub.isConnected                     "LS_tenantId" = $ls.tenantId                     "BackingPortGroup" = $portgroup.name                     "VirtualMachine" = $virtualmachine.name                     "VirtualMachineNICname" = $VMdetails.name                     "VirtualMachineNICmac" = $VMdetails.MacAddress                  } # END pscustomobject             }         }     } $getNsxLogicalRouter | export-csv $PathExportNsxLogicalRouter -NoTypeInformation $output | export-csv $PathExportNsxLogicalSwitch -NoTypeInformation } end {    Disconnect-NsxServer } II. Migration 1. Create a new non overlapping Segment Range using Postman (Body > raw): POST https://10.10.10.40/api/2.0/vdn/config/segments <segmentRange> <name>DATACENTER</name> <begin>10001</begin> <end>20000</end> </segmentRange> # Note the segment range “id” (lets call it newRangeId) returned in response payload. 2. # GET segments will also return segment range "id" using Postman: GET https://10.10.10.40/api/2.0/vdn/config/segments it will return <newRangeId> here example output: <segmentRanges>   <segmentRange>     <id>1</id>   <- this is the ID to use in step 4.     <name>5000-5999</name>     <begin>5000</begin>     <end>5999</end>     <isUniversal>false</isUniversal>     <universalRevision>0</universalRevision>   </segmentRange> </segmentRanges> 3. Disconnect Edges, VMs, vNIC from the dvpg (LogicalSwitch) by following the steps bellow: 3.1. Before any deletion every logical switch connection should be write down (VMs, Edges): 3.1.1. Home > Network and Security > Logical Switches > take screenshot of Logical Switch ID, Segment ID, Name > Click on the logical switch > Related Objects > take screenshot of Edge tab, VMs tab 3.1.2. Home > Network and Security > Edge Gateways > Click on the Edge (or Logical Router) > Manage > Settings > Interfaces (take a screenshot & write down the information inside the edit menu) 3.1.3. Based on the logical switch ID go to network port group and take a screenshot of the VMs: Home > Networking > portgroup name > VMs 3.2. Remove and disconnect the related objects: 3.2.1. Home > Network and Security > Logical Switches > Select each logical switch > Related Objects > Actions > Remove VM > Select all the VMs in the list > Remove # DisconnectNic is taken from   PowerNSX module function DisconnectNic {                     param (         $nic,         $WaitTimeout = 90     )                     #See NSX API guide 'Attach or Detach a Virtual Machine from a Logical Switch' for     #how to construct NIC id.     $vmUuid = ($nic.parent | get-view).config.instanceuuid     $vnicUuid = "$vmUuid.$($nic.id.substring($nic.id.length-3))"                     #Construct XML     $xmldoc = New-Object System.Xml.XmlDocument     $xmlroot = $xmldoc.CreateElement("com.vmware.vshield.vsm.inventory.dto.VnicDto")     $null = $xmldoc.AppendChild($xmlroot)     Add-XmlElement -xmlRoot $xmlroot -xmlElementName "objectId" -xmlElementText $vnicUuid     Add-XmlElement -xmlRoot $xmlroot -xmlElementName "vnicUuid" -xmlElementText $vnicUuid     Add-XmlElement -xmlRoot $xmlroot -xmlElementName "portgroupId" -xmlElementText ""                     #Do the post     $body = $xmlroot.OuterXml     $URI = "/api/2.0/vdn/virtualwires/vm/vnic"     if ( $confirm ) {         $message  = "Disconnecting $($nic.Parent.Name)'s network adapter from a logical switch will cause network connectivity loss."         $question = "Proceed with disconnection?"                         $choices = New-Object Collections.ObjectModel.Collection[Management.Automation.Host.ChoiceDescription]         $choices.Add((New-Object Management.Automation.Host.ChoiceDescription -ArgumentList '&Yes'))         $choices.Add((New-Object Management.Automation.Host.ChoiceDescription -ArgumentList '&No'))                         $decision = $Host.UI.PromptForChoice($message, $question, $choices, 1)     }     else { $decision = 0 }     if ($decision -eq 0) {         Write-Progress -Activity "Processing" -Status "Disconnecting $vnicuuid from logical switch"         $response = invoke-nsxwebrequest -method "post" -uri $URI -body $body -connection $connection         Write-Progress -Activity "Processing" -Status "Disconnecting $vnicuuid from logical switch" -Completed                         $job = [xml]$response.content         $jobId = $job."com.vmware.vshield.vsm.vdn.dto.ui.ReconfigureVMTaskResultDto".jobId                         Wait-NsxGenericJob -Jobid $JobID -Connection $Connection -WaitTimeout $WaitTimeout -FailOnTimeout:$FailOnTimeout                     } } #vCenter Connection and Path to file $VIServer = "YOURCLOUDVCR01.local" $connection = Connect-NSXServer -vCenterServer $VIServer $defaultNsxConnection = $connection $defaultViServer = $connection.viConnection # Point to the CSV file generated from the script above !!! $Import = import-csv C:\Support\Scripts\NSX\NSXReport.csv # Put the current Virtual Wire you are working on $virtualwire = "virtualwire-01" $pathToVMList = $Import | where {($_.LS_ObjectID -eq $virtualwire) -and ($_.VirtualMachine -notlike "vse-*")} # disconnect VM from Logical Switch (there is a 100 sec timeout) foreach ($vm in $pathToVMList){ $VirtualMachineNic = get-vm $VM.VirtualMachine | Get-NetworkAdapter | where {$_.NetworkName -eq $VM.BackingPortGroup} DisconnectNic -nic $VirtualMachineNic  -WaitTimeout 100 } 3.2.2. Home > Network and Security > NSX Edges > Double Click on the Edge (or Logical Router) > Manage > Settings > Interfaces (take the name of the logical switch) usually vNIC 1 > Select (radio button) > Disconnect > Confirm "Yes" > wait till Pending Job finish. When disconnecting edges with High Availability configured, do remember to check and ensure HA is not configured on a logical switch also. (if the HA configuration is vNic "Any" there is no need to change anything) Note: if you have only one connected interface you should connect another one and then disconnect the original one which should be migrated. After the migration connect back the original one and delete the temporary one. 4. Move each logical switch from the old segment range to new segment range. This API needs virtualwire-id and rangeId as inputs which can be taken from the get-NSXinfo report. API payload is empty (on success the status code of the request will be "200 OK"): PUT https://10.10.10.40/api/2.0/vdn/virtualwires/virtualwire-100/segmentreconfig/<newRangeId> === Try this in case of an error: POST "https://10.10.10.40/api/2.0/vdn/virtualwires/virtualwire-40/backing?action=remediate === 5. ONLY for Logical routers: 5.1. POST https://10.10.10.40/api/4.0/edges/{edge-id}?action=vdridreconfig&vdnRangeId=<newRangeId> Output: 204 (No Content) 6. Go to Home > Network and Security > NSX Edges > Double Click on the Edge > Manage > Settings > Interfaces and then reconnect the interface that was disconnected (wait till Pending Job finish) 7. Redeploy the migrated edge/logical router 8. Check if the new configuration for each logical router is pushed to the host with net-vdr  - "net-vdr -L -l edge-113 more" #http://www.enterprisedaddy.com/2018/04/how-to-execute-script-remotely-on-esxi-hosts/ # C:\Support\plink.exe is needed. # add info $root = "root" $Passwd = "  add password here   " $esxlist = " add servers here", "add servers here" $edge = "edge-123" # "edge-100" # work $cmd = "net-vdr -L -l $edge" $plink = "echo y | C:\Support\plink.exe" $remoteCommand = '"' + $cmd + '"' $outResult = foreach ($esx in $esxlist) {     Connect-VIServer -Server $esx -User  $root -Password $Passwd > $null     # Write-Host -Object "starting ssh services on $esx"     $sshstatus = Get-VMHostService  -VMHost $esx | Where-Object { $psitem.key -eq "tsm-ssh" }     if ($sshstatus.Running -eq $False) {         Get-VMHostService | Where-Object { $psitem.key -eq "tsm-ssh" } | Start-VMHostService     }     # Write-Host -Object "Executing Command on $esx"     $output = $plink + " " + "-batch -ssh" + " " + $root + "@" + $esx + " " + "-pw" + " " + $Passwd + " " + $remoteCommand     $message = Invoke-Expression -command $output     [PSCustomObject]@{         Name = $esx         Vxlan = ($message | Select-String -Pattern "Vxlan:").ToString().split("Vxlan:")[-1]     }        Disconnect-VIServer -Server $esx -Confirm:$false } $outResult 9. Home > Network and Security > Logical Switches > Select each logical switch > Related Objects > Actions > Add VM > Search for the name of the VM > Select the VM > Click the right arrow > Next > Select the appropriate network adapter > Next > Finish # connect foreach ($vm in $pathToVMList){ $VirtualMachineNic = get-vm $VM.VirtualMachine | Get-NetworkAdapter | where {($_.MacAddress -eq $VM.VirtualMachineNICmac) -and ($_.Name -eq $VM.VirtualMachineNICname)} Connect-NsxLogicalSwitch -NetworkAdapter $VirtualMachineNic -LogicalSwitch (Get-NsxLogicalSwitch -Name $VM.LS_Name) -WaitTimeout 100 } 10. After we migrate all Logical Switches and routers (on success the status code of the request will be "200 OK"): DELETE https://10.10.10.40/api/2.0/vdn/config/segments/<oldRangeId> 11. Enable the integration between the vCD and the vCenter: login to vCD > Manage & Monitor > vCenters > Right click on the vCenter server > Enable 12. Change the cluster DRS from "Manual" to "Fully Automated" =========================================== Backout plan: 1. Login vCD (https://yourowncloud.com): Manage & Monitor > vCenters > Right click on the vCenter server > Disable 2. Login to https://YOURCLOUDVCR01.local 3. Disconnect Edges, VMs, vNIC from the dvpg (LogicalSwitch) by following the steps bellow: 3.1. Before any deletion every logical switch connection should be write down (VMs, Edges): 3.1.1. Home > Network and Security > Logical Switches > take screenshot of Logical Switch ID, Segment ID, Name > Click on the logical switch > Related Objects > take screenshot of Edge tab, VMs tab 3.1.2. Home > Network and Security > Edge Gateways > Click on the Edge (or Logical Router) > Manage > Settings > Interfaces (take a screenshot & write down the information inside the edit menu) 3.1.3. Based on the logical switch ID go to network port group and take a screenshot of the VMs: Home > Networking > portgroup name > VMs 3.2. Remove and disconnect the related objects: 3.2.1. Home > Network and Security > Logical Switches > Select each logical switch > Related Objects > Actions > Remove VM > Select all the VMs in the list > Remove # DisconnectNic is taken from PowerNSX module function DisconnectNic {                     param (         $nic,         $WaitTimeout = 90     )                     #See NSX API guide 'Attach or Detach a Virtual Machine from a Logical Switch' for     #how to construct NIC id.     $vmUuid = ($nic.parent | get-view).config.instanceuuid     $vnicUuid = "$vmUuid.$($nic.id.substring($nic.id.length-3))"                     #Construct XML     $xmldoc = New-Object System.Xml.XmlDocument     $xmlroot = $xmldoc.CreateElement("com.vmware.vshield.vsm.inventory.dto.VnicDto")     $null = $xmldoc.AppendChild($xmlroot)     Add-XmlElement -xmlRoot $xmlroot -xmlElementName "objectId" -xmlElementText $vnicUuid     Add-XmlElement -xmlRoot $xmlroot -xmlElementName "vnicUuid" -xmlElementText $vnicUuid     Add-XmlElement -xmlRoot $xmlroot -xmlElementName "portgroupId" -xmlElementText ""                     #Do the post     $body = $xmlroot.OuterXml     $URI = "/api/2.0/vdn/virtualwires/vm/vnic"     if ( $confirm ) {         $message  = "Disconnecting $($nic.Parent.Name)'s network adapter from a logical switch will cause network connectivity loss."         $question = "Proceed with disconnection?"                         $choices = New-Object Collections.ObjectModel.Collection[Management.Automation.Host.ChoiceDescription]         $choices.Add((New-Object Management.Automation.Host.ChoiceDescription -ArgumentList '&Yes'))         $choices.Add((New-Object Management.Automation.Host.ChoiceDescription -ArgumentList '&No'))                         $decision = $Host.UI.PromptForChoice($message, $question, $choices, 1)     }     else { $decision = 0 }     if ($decision -eq 0) {         Write-Progress -Activity "Processing" -Status "Disconnecting $vnicuuid from logical switch"         $response = invoke-nsxwebrequest -method "post" -uri $URI -body $body -connection $connection         Write-Progress -Activity "Processing" -Status "Disconnecting $vnicuuid from logical switch" -Completed                         $job = [xml]$response.content         $jobId = $job."com.vmware.vshield.vsm.vdn.dto.ui.ReconfigureVMTaskResultDto".jobId                         Wait-NsxGenericJob -Jobid $JobID -Connection $Connection -WaitTimeout $WaitTimeout -FailOnTimeout:$FailOnTimeout                     } } #vCenter Connection and Path to file $VIServer = "YOURCLOUDVCR01.local" $connection = Connect-NSXServer -vCenterServer $VIServer $defaultNsxConnection = $connection $defaultViServer = $connection.viConnection # csv file from get-NSXinfo $Import = import-csv C:\Support\Scripts\NSX\NSXReport.csv $virtualwire = "virtualwire-60" $pathToVMList = $Import | where {($_.LS_ObjectID -eq $virtualwire) -and ($_.VirtualMachine -notlike "vse-*")} # disconnect foreach ($vm in $pathToVMList){ $VirtualMachineNic = get-vm $VM.VirtualMachine | Get-NetworkAdapter | where {$_.NetworkName -eq $VM.BackingPortGroup} DisconnectNic -nic $VirtualMachineNic  -WaitTimeout 100 } 3.2.2. Home > Network and Security > NSX Edges > Double Click on the Edge (or Logical Router) > Manage > Settings > Interfaces (take the name of the logical switch; e.g. dvs.....) usually vNIC 1 > Select (radio button) > Disconnect > Confirm "Yes" > wait till Pending Job finish. When disconnecting edges with High Availability configured, do remember to check and ensure HA is not configured on a logical switch also. (if the HA configuration is vNic "Any" there is no need to change anything) 4. Move each logical switch from the old segment range to new segment range. This API needs virtualwire-id and rangeId as inputs which can be taken from the get-NSXinfo report. API payload is empty (on success the status code of the request will be "200 OK"): PUT https://10.10.10.40/api/2.0/vdn/virtualwires/virtualwire-100/segmentreconfig/<newRangeId> === Try this in case of an error: POST "https://10.10.10.40/api/2.0/vdn/virtualwires/virtualwire-40/backing?action=remediate === 5. ONLY for Logical routers: 5.1. POST https://10.10.10.40/api/4.0/edges/{edge-id}?action=vdridreconfig&vdnRangeId=<newRangeId> 6. Go to Home > Network and Security > NSX Edges > Double Click on the Edge > Manage > Settings > Interfaces and then reconnect the interface that was disconnected (wait till Pending Job finish) 7. Redeploy the migrated edge/logical router 8. Check if the new configuration for each logical router is pushed to the host "net-vdr -L -l edge-113 more" #http://www.enterprisedaddy.com/2018/04/how-to-execute-script-remotely-on-esxi-hosts/ # C:\Support\plink.exe is needed. # add info $root = "root" $Passwd = "  add password here   " $esxlist = " add servers here", "add servers here" $edge = "edge-123" # "edge-117" # work $cmd = "net-vdr -L -l $edge" $plink = "echo y | C:\Support\plink.exe" $remoteCommand = '"' + $cmd + '"' $outResult = foreach ($esx in $esxlist) {     Connect-VIServer -Server $esx -User  $root -Password $Passwd > $null     # Write-Host -Object "starting ssh services on $esx"     $sshstatus = Get-VMHostService  -VMHost $esx | Where-Object { $psitem.key -eq "tsm-ssh" }     if ($sshstatus.Running -eq $False) {         Get-VMHostService | Where-Object { $psitem.key -eq "tsm-ssh" } | Start-VMHostService     }     # Write-Host -Object "Executing Command on $esx"     $output = $plink + " " + "-batch -ssh" + " " + $root + "@" + $esx + " " + "-pw" + " " + $Passwd + " " + $remoteCommand     $message = Invoke-Expression -command $output     [PSCustomObject]@{         Name = $esx         Vxlan = ($message | Select-String -Pattern "Vxlan:").ToString().split("Vxlan:")[-1]     }        Disconnect-VIServer -Server $esx -Confirm:$false } $outResult 9. Home > Network and Security > Logical Switches > Select each logical switch > Related Objects > Actions > Add VM > Search for the name of the VM > Select the VM > Click the right arrow > Next > Select the appropriate network adapter > Next > Finish # connect foreach ($vm in $pathToVMList){ $VirtualMachineNic = get-vm $VM.VirtualMachine | Get-NetworkAdapter | where {($_.MacAddress -eq $VM.VirtualMachineNICmac) -and ($_.Name -eq $VM.VirtualMachineNICname)} Connect-NsxLogicalSwitch -NetworkAdapter $VirtualMachineNic -LogicalSwitch (Get-NsxLogicalSwitch -Name $VM.LS_Name) -WaitTimeout 100 } 10. Enable the integration between the vCD and the vCenter: login to vCD > Manage & Monitor > vCenters > Right click on the vCenter server > Enable 11. Change the cluster DRS from "Manual" to "Fully Automated" =========================================== Impact: During the migration for each logical switch there will be a short (5-10 minutes) disconnection of the networking for all Edges, Logical Routers and VMs. All related networks which are in the current Logical Segment Pool will lose connection to the migrated logical switch which is in the new Segment ID Pool. VMs and Edges: is equal to unplug the network cable from a physical server. =========================================== Test Details: 1. Log in into https://YOURCLOUDVCR01.local 2. Go to Network & Security 3. Check the status of the Logical Switch (Logical Switches section) 4. Check the status of the Edges connected to the logical switch (Edge section) 5. Based on the information extracted before the change check the status of the VMs connected to the Logical switch 6. Check the options are in place after refreshing the vSphere Web Client. 7. Go to vCD: https://yourowncloud.com and check the status of the Orgs 8. Go to vCD and check the logs under System 9. Go to vCD: check Stranded Items, Switches & Port Groups, Storage Policies, Datastores, Hosts, Resource Pools, vCenters, Network Pools, External Networks, Edge Gateways, Organization VDCs, Provider VDCs, Cloud Cells, Organizations 10. Check if there are errors/warnings on cluster level for the tenant which was migrated 11. Check each host which is part of the tenant cluster if there are errors in: /var/log/vmkernel.log (Use Log Insight) 12. Manually move several VMs in the vCenter server and check if there are warnings/errors in the tenant cluster 13. Wait DRS to automatically move some VMs from one host to another and check for warnings/errors in the tenant cluster 14. Check again the status of the VMs and the Edges inside the vCD