Highlighted
Enthusiast
Enthusiast

NSX-Manager Error: Component NSX is not Yet in Running phase, current phase stopped.

Dear All,

My NSX manager giving the subject error and NSX management service is also stopped. When i try to start the service manually but it fails. I have also check the nsx-manager logs through "show log manager reverse" command but could not find the error.

0 Kudos
13 Replies
Highlighted
Enthusiast
Enthusiast

HI POFIT

what version of NSX do you run ? did you upgraded from one version to another version ?

Cheers

0 Kudos
Highlighted
Expert
Expert

If  the Management service stopped after Upgrade, there could be upgrade path that is not supported directy:

https://kb.vmware.com/s/article/2135476

After upgrading VMware NSX for vSphere 6.1.5 to 6.2.0, you experience these symptoms:

  • In the NSX Manager Web User Interface (UI), the NSX Management Service status is Stopped
  • Clicking the Start button to restart the NSX Management Service returns the status back to Stopped
  • When you navigate to vCenter Server > Networking & Security > Installation, you see the error:

    Could not establish communication with NSX Manager.

Also is the error on the Vcenter > Networking and Security are there any errors different than above could not establish communication error?

0 Kudos
Highlighted
Enthusiast
Enthusiast

Yes This is correct

but not only this can be the reason of the issue , so we need version of NSX and if the upgrade was done or not from which to which.

0 Kudos
Highlighted
Enthusiast
Enthusiast

     Dear All,

I have not upgraded the NSX version and i am using the NSX version 6.2. The vcenter also giving the srror when i click on networking and security dashboad "No NSX manager available. Verify current user has role assigned on NSX Manager. Actually i have reboot the Vcenter server and NSX manager due to space issue after that i am facing this issue.

0 Kudos
Highlighted
Expert
Expert

Are there any errors on NSX manager logs, it is possible to see with NSX Manager CLI show log command:

show log manager follow

show log manager reverse

If it is a Nested environment this could be helpful:

https://www.vmguru.com/2016/01/ravello-systems-vmware-nsx-6-2-management-service-stuck-in-starting/

1

2

3

manager# show log manager reverse

2016-01-03 14:46:00.548 GMT  INFO localhost-startStop-1 VsmServletContextListener:75 - NSX Status : STOPPED

        ... 129 more

The vNIC of the NSX Manager is standardised on VMXNet3, which is fine under normal circumstances. The trick is to set the vNIC type to e1000. When you deploy the NSX Manager 6.2.1 with a e1000 vNIC, all required services boot up like you’d expect them to

0 Kudos
Highlighted
Expert
Expert

  • Alsow show filesystem could be helpful, as the storage or filesystem usage is 100 %, it may be necessary to purge log system command to free up space.

https://docs.vmware.com/en/VMware-NSX-for-vSphere/6.3/com.vmware.nsx.upgrade.doc/GUID-E8BA0AB5-9948-...

The link is from the Upgrade Guide, but it could be used as health of NSX Manager:

Validate the NSX Manager file system usage, and perform a cleanup if file system usage is at 100 percent.

  1. Login to NSX Manager and run show filesystems to show the filesystem usage.
  2. If the usage is 100 percent, run the purge log manager and purge log system commands.
  3. Reboot the NSX Manager appliance for the log cleanup to take effect.

  • From the Remote Console of the NSX Manager reboot command, are there any errors during the bootup cycle simlar to Unexpected Inconsistency or Filesystem errors?

If during the bootup https://blah.cloud/virtualisation/recovering-nsx-manager-corrupt-filesystem/

https://docs.vmware.com/en/VMware-NSX-T/2.0/com.vmware.nsxt.admin.doc/GUID-E2AC3DFB-7001-4873-872A-D...

You can reboot NSX Manager with a CLI command to recover from critical errors.

Procedure

  1. Log in to the CLI of NSX Manager.
  2. Run the reboot command. For example,
    nsx-manager> reboot Are you sure you want to reboot (yes/no): y
0 Kudos
Highlighted
Enthusiast
Enthusiast

HI

did the registration tab is used same username and password ?

do you use full format i.e @domain.com ?

What kind of user do you use localos, DC , vsphere.local.?

Regards Dmitri

0 Kudos
Highlighted
Enthusiast
Enthusiast

Dear,

I have run both the commands,

#show log manager reverse

#show log manager fellow

and i have find the following errors in the logs.

SQLException for SQL [???]; SQL state [null]; error code [0] cannot open connection nested exception is org.hibernate.exception.jdbcconnectionexception

ERROR NVPStatusCheck ControllerServiceImpl - Checking controller node stats failed

WARN HostStatusServiceInitThread HostStatusService Iml : 141 Failed to get initail host ID list from Network Preparation Service . Could not open JPA EntityManger for transaction; nested exception is javax,persistence.persistenceException; org.hibernate.exception.GenericJBDCException cnnot open connection.

0 Kudos
Highlighted
Enthusiast
Enthusiast

can you put full java exception code!

Regards Dmitri

0 Kudos
Highlighted
Contributor
Contributor

Are you seing these error messages after restarting the NSX Manager?

What is the NSX version?

0 Kudos
Highlighted
Expert
Expert

The  JDBC errors seem to be related with the PostgreSQL Db of the NSX Manager, which is internal. On the NSX Troubleshooting document there are no detailed explanation of how to check if the Internal DB is healthy, mostly concentrating on the outbound tcp connections. If the vPostgres service is  also not running on the Summary page, then it may be concentrated on troubleshooting of first this service, since NSX Management service is the main service, which may depend on other services to start.

One workaraound could be to restore NSX manager backup taken before the problem, but this would not solve the problem itself and may recreate if the root cause is not clear.

SQLException for SQL [???]; SQL state [null]; error code [0] cannot open connection nested exception is org.hibernate.exception.jdbcconnectionexception

This may indicate a problem with JDBC connection to the internal PostGreSQL database of the NSX Manager. On the same NSX Manager Summary Page what is the Status of the vPostgres service?

ERROR NVPStatusCheck ControllerServiceImpl - Checking controller node stats failed

It is possible that NSX Manager - Controller Cluster connectivity problem. NSX Manager cannot form TCP Connection to the cluster.

On NSx Manager CLI debug connection does the NSX Manager try to establish connection to Controller Cluster members or Vcenter? This may not be because NSX Management service is the main service

https://docs.vmware.com/en/VMware-NSX-for-vSphere/6.3/com.vmware.nsx.troubleshooting.doc/GUID-16BFCF...

Perform Packet Capture on NSX Manager to View Connections

Use the debug packet command: debug packet [capture|display] interface interface filter

The interface name on NSX Manager is mgmt.

The filter syntax follows this form: "port_80_or_port_443"

The command runs in privileged mode only. To enter privileged mode, run the enable command and provide the admin password.

Packet capture example:

nsxmgr# en

nsxmgr# debug packet display interface mgmt port_80_or_port_443 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on mgmt, link-type EN10MB (Ethernet), capture size 262144 bytes 23:40:25.321085 IP 192.168.210.15.54688 > 192.168.210.22.443: Flags [P.], seq 2645022162:2645022199, ack 2668322748, win 244, options [nop,nop,TS val 1447550948 ecr 365097421], length 37 ...

http://www.virten.net/2016/05/vmware-nsx-6-component-communication-diagram/

Component_Communication_Diagram.png

NSX Manager > NSX Controller 443/TCP (Controller to Manager Communication)

Is the controller cluster if checked  from the Controller CLI formed and working properly?

http://www.vmwarearena.com/vmware-nsx-installation-part-5-checking-nsx-controller-status/

WARN HostStatusServiceInitThread HostStatusService Iml : 141 Failed to get initail host ID list from Network Preparation Service . Could not open JPA EntityManger for transaction; nested exception is javax,persistence.persistenceException; org.hibernate.exception.GenericJBDCException cnnot open connection.

This error again seems to to be Db related like before messages. The Web server of the NSX Manager and PostGresql may need to be troubleshoot

0 Kudos
Highlighted
Enthusiast
Enthusiast

Please see this KB VMware Knowledge Base

also there is a possibility, depend of the version, that NSX manager cant send API call to controllers because of max connection , do you have any firewall between NSX Manager and controllers ?

Use API call directly to controllers to verify if they respond

Regards Dmitri

0 Kudos
Highlighted
Enthusiast
Enthusiast

Dear All,

Dear i am execute the command "debug packet display interface mgmt port_80_or_port_443" and it is communicating one the controller three controllers i have configured. I can also access through ssh all the controllers. Moreover vPostgres and RabbitMQ sevices are running. I am also made one change. NSX Manager network card giving warning on Network adapter VMXNET type. The warning gone when i change it to e1000. But on both cases the nsx manager services is not running. 

0 Kudos