VMware Networking Community
Marmotte94
Enthusiast
Enthusiast

Error Message on NSX-t 3

Hi guys,

I have theses errors message on my nsx-t from today but I don't know how can I solve it.

Can you help me please ?

Error message :

1/ Failed to fetch System details. Please contact the administrator. Error: {1}

2/ Logical Router list failed: Index out of sync, please resync via 'start search resync manager'.

Thank you regards,

Please, visit my blog http://www.purplescreen.eu/
Reply
0 Kudos
16 Replies
daphnissov
Immortal
Immortal

What did the logs say when you examined them?

Reply
0 Kudos
nachogonzalez
Commander
Commander

Hey, hope you are doing fine:

Where do you get the error? 
Are you able to log in to NSX Manager?

Can you confirm if NTP and DNS are working correctly?

warm regards

Reply
0 Kudos
Marmotte94
Enthusiast
Enthusiast

Where do you get the error?

The first error appear on "System Overview" and some overview. And they didn't get any data on it.

The second is on logical "Tier-x Logical Routers " view.


Are you able to log in to NSX Manager?

I can't get support bundle as well.

Can you confirm if NTP and DNS are working correctly?

Can you tell me where can I check it please ? Sorry I'm a beginner with NSX-t. I have just set it this weekend and I did BGP configuration as well.

Please, visit my blog http://www.purplescreen.eu/
Reply
0 Kudos
nachogonzalez
Commander
Commander

Where do you get the error?

The first error appear on "System Overview" and some overview. And they didn't get any data on it.

The second is on logical "Tier-x Logical Routers " view.
Please attach some screenshots.


Are you able to log in to NSX Manager?

I can't get support bundle as well

What happens if you log in to the NSX manager via GUI and go to system overview? can you share a screenshot?
What happens if you log in via SSH or console to NSX manager cluster and run get cluster config?

View the Configuration and Status of the NSX Manager Cluster

Can you confirm if NTP and DNS are working correctly?

Can you tell me where can I check it please ? Sorry I'm a beginner with NSX-t. I have just set it this weekend and I did BGP configuration as well.
When you deployed the NSX manager cluster you should've set an NTP and DNS server.
Please log in via SSH console to nsx and run NTPQ queries

Reply
0 Kudos
Lalegre
Virtuoso
Virtuoso

Hey Marmotte94​,

Is your installation a cluster? Could you please check if all the services are up into the appliances with the next command: get services. Check the status of the NSX-T Cluster running: get cluster status

Also please connect as root using SSH and run df -h on all the appliances to verify you have enough free space.

Finally, try to login to every appliance individually using IP and FQDN (Same with VIP in case cluster) to see if the issues does not replicates on one of those cases.

Reply
0 Kudos
Marmotte94
Enthusiast
Enthusiast

Hi Guys,

Thank you for your help but I have an issue with sphere 7. I can not enter my password because in my vSphere 7 environment I can not enter some caractere like ")" from web console of the manager NSX-t. So I can't log in as root on console. I can't edit GRUB as well (Configure NSX-T Data Center to Display the GRUB Menu at Boot Time ) because of log-in as root. And Invoke script doesn't work as well.

So I must deploy another appliance. I think the mapping of keyboard is not very good in sphere 7. It's a bug

Thank you

Regads,

Please, visit my blog http://www.purplescreen.eu/
Reply
0 Kudos
mauricioamorim
VMware Employee
VMware Employee

Can you access using SSH using admin user? If you can then with the command "st eng" you can jump to root shell.

Reply
0 Kudos
Marmotte94
Enthusiast
Enthusiast

Hi,

I succeeded to connect to remote console with this tools : govmomi/USAGE.md at master · vmware/govmomi · GitHub

I checked DNS and Time and all seem good. I always error from NSX-t overview. You can check me previous message with screenshot.

Thank you,

Please, visit my blog http://www.purplescreen.eu/
Reply
0 Kudos
Marmotte94
Enthusiast
Enthusiast

Here screenshot for NTP and DNS configuration

Please, visit my blog http://www.purplescreen.eu/
Reply
0 Kudos
Marmotte94
Enthusiast
Enthusiast

This is log file from /var/log/syslog

<182>1 2020-09-22T17:16:34.534Z nsxmanager NSX 6227 - [nsx@6876 audit="true" comp="nsx-manager" level="INFO" reqId="7fec8316-da7f-4773-90fd-4eff73d77205" subcomp="manager"] UserName="admin", ModuleName="NsxHome", Operation="ReadFabricAggregateStatus", Operation status="success"

<182>1 2020-09-22T17:16:34.541Z nsxmanager NSX 6477 - [nsx@6876 audit="true" comp="nsx-manager" level="INFO" reqId="738fb01b-a383-4fea-a422-9656e9b17052" subcomp="policy" username="admin"] UserName="admin", ModuleName="AAA", Operation="GetUserFeaturePermissions", Operation status="success"

<182>1 2020-09-22T17:16:34.566Z nsxmanager NSX 6227 - [nsx@6876 audit="true" comp="nsx-manager" level="INFO" reqId="cd893e62-cd62-43d9-a228-19a3d957123c" subcomp="manager" username="admin"] UserName="admin", ModuleName="Search", Operation="GroupBySearch", Operation status="failure", New value=[{"query":"resource_type:Alarm AND entity_resource_type: ClusterNodeConfig AND status: OPEN AND node_id:218c0842-9623-4d86-654c-b5c0875cac4b","group_count":10,"group_by_field":"severity","exclude_internal_types":false}]

<182>1 2020-09-22T17:16:34.566Z nsxmanager NSX 6227 - [nsx@6876 audit="true" comp="nsx-manager" level="INFO" reqId="cd893e62-cd62-43d9-a228-19a3d957123c" subcomp="manager"] UserName="admin", ModuleName="NsxQuery", Operation="ReadClustersOverallStatus", Operation status="failure", New value=[{"role":"NodeClusterRoleConfig"}]

<182>1 2020-09-22T17:16:34.566Z nsxmanager NSX 6227 - [nsx@6876 audit="true" comp="nsx-manager" level="INFO" subcomp="manager"] UserName:'admin' ModuleName:'nsx-search' Operation:'GET@/api/v1/ui-controller/clusters-overall-status'Operation status: 'failure' Error: Index out of sync, please resync via 'start search resync manager'.

<179>1 2020-09-22T17:16:34.569Z nsxmanager NSX 6477 - [nsx@6876 comp="nsx-manager" errorCode="MP513002" level="ERROR" subcomp="policy"] REST API failed: http://127.0.0.1:7440/nsxapi/api/v1/ui-controller/clusters-overall-status?role=NodeClusterRoleConfig GET  with status 400 BAD_REQUEST

<180>1 2020-09-22T17:16:34.571Z nsxmanager NSX 6477 - [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="policy"] API to fetch NSX nodes failed at home page. Error: com.vmware.nsx.management.common.exceptions.InvalidArgumentException: Failed to fetch System details. Please contact the administrator. Error: {1}

<179>1 2020-09-22T17:16:34.571Z nsxmanager NSX 6477 - [nsx@6876 comp="nsx-manager" errorCode="MP513002" level="ERROR" subcomp="policy"] Cannot fetch system details: Failed to fetch System details. Please contact the administrator. Error: {1}

<182>1 2020-09-22T17:16:34.571Z nsxmanager NSX 6477 - [nsx@6876 audit="true" comp="nsx-manager" level="INFO" reqId="738fb01b-a383-4fea-a422-9656e9b17052" subcomp="policy"] UserName="admin", ModuleName="NsxHome", Operation="ReadSystemAggregateStatus", Operation status="failure"

<182>1 2020-09-22T17:16:34.571Z nsxmanager NSX 6477 - [nsx@6876 audit="true" comp="nsx-manager" level="INFO" subcomp="policy"] UserName:'admin' ModuleName:'Policy' Operation:'GET@/api/v1/ui-controller/system-aggregate-status'Operation status: 'failure' Error: Failed to fetch System details. Please contact the administrator. Error: {1}

<180>1 2020-09-22T17:16:36.190Z nsxmanager NSX 6477 - [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="policy"] ConnectionFuture completed with error io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /127.0.0.1:9831. Retrying...

Please, visit my blog http://www.purplescreen.eu/
Reply
0 Kudos
Lalegre
Virtuoso
Virtuoso

It seems that is failing to get the status of the cluster, please run the commands I posted above and share with us the output.

Reply
0 Kudos
Marmotte94
Enthusiast
Enthusiast

Hi,

This is the Output as you need.

Thank you for your help.

Please, visit my blog http://www.purplescreen.eu/
Reply
0 Kudos
Lalegre
Virtuoso
Virtuoso

Everything seems fine regarding those outputs, however in the first capture I can see that it displays something like nsXXXXX and bunch of number as the hostname but from the SSH it shows the correct hostname. I just checked on my LAB and i do not have it like that, did you put the correct value on hostname when deploying?

Also could you please check that there is no DNS A or PTR record conflicting with your installation?

Reply
0 Kudos
Marmotte94
Enthusiast
Enthusiast

lol. Sorry this is the good screenshot :smileysilly:

Please, visit my blog http://www.purplescreen.eu/
Reply
0 Kudos
Lalegre
Virtuoso
Virtuoso

Have you actually try the command that is shown in the error: 'start search resync manager',

Also please try the next API Call:

  • POST https://<nsx-mgr>/api/v1/cluster/node?action=repo_sync

And after that run the next API call to get the node-id:

  • GET https://<nsx-mgr>/api/v1/cluster/nodes/

After getting the node-id check the status of the sync:

  • GET https://<nsx-mgr>/api/v1/cluster/nodes/Node_id_previous_Step/repo_sync/status

This is supposed to be run for a NSX-T cluster but I think it will work for only one node.

Reply
0 Kudos
goooo1
Contributor
Contributor

Just yesterday, I restored the configuration of nsx-t manager once, Version 3.2 ,the same error as yours.

I don't know how to fix it

goooo1_0-1640157655256.png

 

Reply
0 Kudos