VMware Networking Community
niceguy001
Enthusiast
Enthusiast

NSX-T manager error_code 101?

i deployed a nsx-t 2.4 manager on KVM with "medium" type, it worked just fine until i installed second manager(on another KVM host) and join it to the first one.

they all display the following messages on the web browser after i type in the admin account and password to login, as shown below:

gtjhtjnbv.jpg

i could not figure out why this error happen and it just can't be fixed after i restart services such as "ui-service" in the manager or even reboot the manager.

does anyone know why? the "manager.log" and "syslog" in the manager appliance did not show messages which is truly helpful.

Tags (2)
Reply
0 Kudos
8 Replies
abhishekkunal51
Contributor
Contributor

Are you able to reach from Manager1 to Manager 2 ?

Can you check the logs on Primary Manager

Reply
0 Kudos
niceguy001
Enthusiast
Enthusiast

hi abhishekkunal5125,

the first manager is able to ping the second one and the second manager can also ping to the first one.

in the primary manager, I used the command "get log-file" to retrieve messages,

the messages in syslog is shown below:

Capture.JPG

the messages in manager.log is shown below:

Capture1.JPG

the messages in manager.log is shown below:

napi.root.cluster.cbm_response WARNING Connection to cbm refused or timed out: 111

and the messages in auth.log is shown below, approximately:

auth.JPG

i guess these information only help a little bit on troubleshooting this error-code-101 issue ?

Reply
0 Kudos
shazbot1
Contributor
Contributor

I had this issue when one of my techs deployed multiple managers at the same time without waiting for them to come up and join the cluster one at a time.  Not sure if this would work for you but we ended up powering them all down, then powering them up in sequence with plenty of time in between.  After about 60 minutes they appeared to sync up and the error code went away.  Granted this is on older, slower hardware.

Reply
0 Kudos
niceguy001
Enthusiast
Enthusiast

shazbot1​ thanks for the suggestioin!

however, shut down and boot again didn't work for my manager so I chose to gave up on this issue and pretend it as an unknown bug.

Reply
0 Kudos
freemanhill
VMware Employee
VMware Employee

I have same quesion , I install a nsx-t 2.4.1 in one cloud , for limited storage , only build one mananger/controller . then install edge and pks ops .

After a storage full issue , restart nsx-t manager  show the same error in web UI/

In the syslog for nsx-manager ; error include:

80>1 2019-09-11T06:22:05.261Z nsxtm1 NSX 3555 - [nsx@6876 comp="nsx-manager" subcomp="node-mgmt" username="root" level="WARNING"] Connection to cbm refused or timed out: 111

<179>1 2019-09-11T06:22:05.263Z nsxtm1.corp.local NSX 5517 - [nsx@6876 comp="nsx-manager" errorCode="MP2101" subcomp="policy"] Request GET http://localhost:7441/api/v1/cluster-manager/config HTTP/1.1 failed, return code is 503

<179>1 2019-09-11T06:22:05.263Z nsxtm1.corp.local NSX 5517 - [nsx@6876 comp="nsx-manager" errorCode="MP2123" subcomp="policy"] Cluster config retrieved from cluster manager is empty.

<180>1 2019-09-11T06:22:05.282Z nsxtm1 NSX 3555 - [nsx@6876 comp="nsx-manager" subcomp="node-mgmt" username="root" level="WARNING"] Connection to cbm refused or timed out: 111

<179>1 2019-09-11T06:22:05.283Z nsxtm1.corp.local NSX 5064 - [nsx@6876 comp="nsx-manager" errorCode="MP2101" subcomp="manager"] Request GET http://localhost:7441/api/v1/cluster-manager/config HTTP/1.1 failed, return code is 503

<179>1 2019-09-11T06:22:05.283Z nsxtm1.corp.local NSX 5064 - [nsx@6876 comp="nsx-manager" errorCode="MP2123" subcomp="manager"] Cluster config retrieved from cluster manager is empty.

Reply
0 Kudos
brandon9neill
Enthusiast
Enthusiast

I know this is old, but since this was one of the few messages that came up when I did a web search for this error, I wanted to add the solution in my environment when I got this error.

It turns out through a bizarre sequence of events, I had lost all my DNS servers.  Once I got that issue fixed and rebooted my NSX Manager, it came back up fine.  This was a lab environment, so I only had one NSX Manager Node.

Reply
0 Kudos
AirCaptain
Contributor
Contributor


@brandon9neill wrote:

I know this is old, but since this was one of the few messages that came up when I did a web search for this error, I wanted to add the solution in my environment when I got this error.

It turns out through a bizarre sequence of events, I had lost all my DNS servers.  Once I got that issue fixed and rebooted my NSX Manager, it came back up fine.  This was a lab environment, so I only had one NSX Manager Node.


Not sure if this would work for you but we ended up powering them all down, then powering them up in sequence with plenty of time in between.  After about

Reply
0 Kudos
nnabelle
Contributor
Contributor


@niceguy001 wrote:

i deployed a nsx-t 2.4 manager on KVM with "medium" type, it worked just fine until i installed second manager(on another KVM host) and join it to the first one.

they all display the following messages on the web browser after i type in the admin account and password to login, as shown below:

gtjhtjnbv.jpg

i could not figure out why this error happen and it just can't be fixed after i restart services such as "ui-service" in the manager or even reboot the manager.

NSX Manager provides a graphical user interface (GUI) and REST APIs for creating, configuring, and monitoring NSX-T Data Center components such as logical switches, logical routers, and firewalls.NSX Manager provides a system view and is the management component of the NSX-T Data Center. For high availability, the NSX-T Data Center supports a management cluster of three NSX Managers.

does anyone know why? the "manager.log" and "syslog" in the manager appliance did not show messages which is truly helpful.


 

greatpeople me

Reply
0 Kudos