VMware Networking Community
m80arm
Enthusiast
Enthusiast
Jump to solution

Missing Logical switch

Hi All,

Having an issue with my NSX test lab and a logical switch that I created that has somewhat disappeared from the GUI and API.  Full details are on my blog here:

M80ARM - Virtualization Warrior: Missing logical switch in NSX GUI

Anyone has any ideas then let me know

Thanks

Michael

Reply
0 Kudos
1 Solution

Accepted Solutions
admin
Immortal
Immortal
Jump to solution

Ok, so I think we've got to the bottom of this. Looks like the request to delete the LS was issued when environment's the only one Controller was not available for whatever reason.

2014-07-02 13:40:55.734 BST  INFO http-nio-127.0.0.1-7441-exec-18 ControllerServiceImpl:692 - remove VNI lswitch=ae65a47b-ad6d-4ff1-ac4c-6b65df2ec7a4

2014-07-02 13:40:55.735 BST  WARN http-nio-127.0.0.1-7441-exec-18 VirtualWireServiceImpl:815 - Ignoring exception : 'all controllers are inactive' during the removal of lSwitch ae65a47b-ad6d-4ff1-ac4c-6b65df2ec7a4.

NSX Manager then proceeded to delete portgroups and whatnot, but stubmbed when it came to deleting Virtual Wire:

2014-07-02 13:40:57.272 BST ERROR http-nio-127.0.0.1-7441-exec-14 VirtualWireServiceImpl:1060 - Virtual Wire virtualwire-6 not found.

2014-07-02 13:40:57.273 BST  WARN http-nio-127.0.0.1-7441-exec-14 RemoteInvocationTraceInterceptor:87 - Processing of VsmHttpInvokerServiceExporter remote call resulted in fatal exception: com.vmware.vshield.vsm.vdn.facade.VdnInventoryFacade.getUiVirtualWire com.vmware.vshield.vsm.exceptions.ObjectNotFoundException: core-services:202:The requested object : virtualwire-6 could not be found. Object identifiers are case sensitive.

What we landed up with is with the situation when Controller still has VNI, but NSX Manager didn't (and no matching portgroups in DVS).

How we recovered:

1) Change the TZ control plane mode to "Multicast", asking it to convert all LSes to the same

2) Delete the Controller that thought it had the VNI

3) Deploy a new Controller, wait for it to come fully online

4) Convert TZ back to Unicast mode, asking it to conver all LSes to the same

Which looks like brought the "lost" LS back into the UI.

Moral of the story: please try to have the recommended number of Controllers (which is three) available. Smiley Happy

View solution in original post

Reply
0 Kudos
9 Replies
GeordyKorte
VMware Employee
VMware Employee
Jump to solution

Hey Micheal,

Try a force sync in the installation tab, that might solve the issue for you

Geordy Korte NSBU (NSX) SE for BeNeLux and Nordics Tweet @gekort
Reply
0 Kudos
m80arm
Enthusiast
Enthusiast
Jump to solution

Geordy,

Thanks for responding, I tried a force sync on all three hosts and also the cluster but the switch still seems to exist.  Very strange issue

Michael

Reply
0 Kudos
admin
Immortal
Immortal
Jump to solution

Hi Michael,

This could be a UI problem.. To rule that out, could you please try to (a) open your vSphere Web Client in a different browser; or (b) force-reload the UI (shift+reload in your browser)?

If this doesn't help, you could try to issue a "repair" against your Transport Zone, as follows:

GET https://10.1.2.41/api/2.0/vdn/scopes

and look for

<vdnScopes>

    <vdnScope>

        <objectId>vdnscope-<X></objectId>

I expect you to see one vdnscope-<X> ("vdnscope-3" in your case), but would be interested to see if more show up.

Then, POST https://10.1.2.41/api/2.0/vdn/scopes/vdnscope-<X>?action=repair

Hope this helps..

Reply
0 Kudos
m80arm
Enthusiast
Enthusiast
Jump to solution

Hi Dmitri,

When trying form a different browser on a different client device I'm still unable to see the logical switch.

Only one vndscope appears when running the command:

<vdnScopes>

<vdnScope>
<objectId>vdnscope-3</objectId>

<objectTypeName>VdnScope</objectTypeName>

<vsmUuid>42260F89-88A8-121D-7087-378A7F8C5419</vsmUuid>

<revision>0</revision>

<type>

<typeName>VdnScope</typeName>

</type>

<name>Transport Zone</name>

<description/>

<clientHandle/>

<extendedAttributes/>

<id>vdnscope-3</id>

<clusters>

<cluster>
<cluster>
<objectId>domain-c36</objectId>

<objectTypeName>ClusterComputeResource</objectTypeName>

<vsmUuid>42260F89-88A8-121D-7087-378A7F8C5419</vsmUuid>

<revision>50</revision>

<type>

<typeName>ClusterComputeResource</typeName>

</type>

<name>TestCluster</name>

<scope>

<id>datacenter-21</id>

<objectTypeName>Datacenter</objectTypeName>

<name>TestLab</name>

</scope>

<clientHandle/>

<extendedAttributes/>

</cluster>
</cluster>
</clusters>

<virtualWireCount>1</virtualWireCount>

<controlPlaneMode>UNICAST_MODE</controlPlaneMode>

</vdnScope>

</vdnScopes>

When running the command:

POST https://10.1.2.41/api/2.0/vdn/scopes/vdnscope-3?action=repair

I get 200 OK and jobdata-5055 retuned in the Response Body.

When checking in the GUI the logical switch still fails to show and when querying the controller directly it still shows VNI 5002 is in use

Michael

Reply
0 Kudos
admin
Immortal
Immortal
Jump to solution

Hi Michael,

Could you try to connect a couple VMs to that logical switch, best if they are on different hosts, and see if they can reach each other? If VMs attach to the logical switch successfully, you should see your hosts connect to the controller (10.1.15.21) - "Connections" and "VTEPs" should show the number of hosts that have VMs connected to the VNI 5002.

Also, how many controllers have you got? Could you please show what "show control-cluster startup-nodes" and "show control-cluster status" says on your controller?

P.S. I see your controller and NSX Manager are on different subnets. It's worth noting that NSX manager has to be able to talk to controllers, and management interfaces on all ESXi hosts, and management interfaces on hosts have to be able to talk to the controllers. Not likely a problem, but just making sure.

P.P.S. Just realised that in your blog you're showing force-syncing routing services, not VXLAN config. See attaches screenshot on where to do it for VXLAN.Untitled.png

Reply
0 Kudos
m80arm
Enthusiast
Enthusiast
Jump to solution

Dmitri,

I can't add the VM's to the logical switch because that switch doesn't actually appear in the GUI and the port group isn't actually available on any of the three hosts in the cluster.  It seems that when I deleted the switch it only half deleted it from the environment and the name is still around somewhere.

I've currently only got one controller.  I've tried removing and re-creating the controllers but this has not worked.  I've added screen shots for the two commands requested.  My network is fully routed so the NSX Manager can communicate with both the controller and all hosts.

Screen Shot 2014-07-04 at 10.53.18.jpeg

Screen Shot 2014-07-04 at 10.53.51.jpeg

I've also tried forcing a re-sync via the VXLAN option but still no joy

Thanks for your help

Michael

Reply
0 Kudos
GeordyKorte
VMware Employee
VMware Employee
Jump to solution

Ok,

Let's get back to the basics... Which version of NSX-V are you running, Version of ESXi, and how did you create the original logical swicth (using gui or api)

Geordy Korte NSBU (NSX) SE for BeNeLux and Nordics Tweet @gekort
Reply
0 Kudos
admin
Immortal
Immortal
Jump to solution

Ok, so I think we've got to the bottom of this. Looks like the request to delete the LS was issued when environment's the only one Controller was not available for whatever reason.

2014-07-02 13:40:55.734 BST  INFO http-nio-127.0.0.1-7441-exec-18 ControllerServiceImpl:692 - remove VNI lswitch=ae65a47b-ad6d-4ff1-ac4c-6b65df2ec7a4

2014-07-02 13:40:55.735 BST  WARN http-nio-127.0.0.1-7441-exec-18 VirtualWireServiceImpl:815 - Ignoring exception : 'all controllers are inactive' during the removal of lSwitch ae65a47b-ad6d-4ff1-ac4c-6b65df2ec7a4.

NSX Manager then proceeded to delete portgroups and whatnot, but stubmbed when it came to deleting Virtual Wire:

2014-07-02 13:40:57.272 BST ERROR http-nio-127.0.0.1-7441-exec-14 VirtualWireServiceImpl:1060 - Virtual Wire virtualwire-6 not found.

2014-07-02 13:40:57.273 BST  WARN http-nio-127.0.0.1-7441-exec-14 RemoteInvocationTraceInterceptor:87 - Processing of VsmHttpInvokerServiceExporter remote call resulted in fatal exception: com.vmware.vshield.vsm.vdn.facade.VdnInventoryFacade.getUiVirtualWire com.vmware.vshield.vsm.exceptions.ObjectNotFoundException: core-services:202:The requested object : virtualwire-6 could not be found. Object identifiers are case sensitive.

What we landed up with is with the situation when Controller still has VNI, but NSX Manager didn't (and no matching portgroups in DVS).

How we recovered:

1) Change the TZ control plane mode to "Multicast", asking it to convert all LSes to the same

2) Delete the Controller that thought it had the VNI

3) Deploy a new Controller, wait for it to come fully online

4) Convert TZ back to Unicast mode, asking it to conver all LSes to the same

Which looks like brought the "lost" LS back into the UI.

Moral of the story: please try to have the recommended number of Controllers (which is three) available. Smiley Happy

Reply
0 Kudos
m80arm
Enthusiast
Enthusiast
Jump to solution

Thanks Dmitri for you help on this.  I've updated my blog post accordingly:

M80ARM - Virtualization Warrior: Missing logical switch in NSX GUI

Michael

Reply
0 Kudos