larstr
Champion
Champion

Unable to deploy NSX-v 6.4.0 controllers

Jump to solution

Hi,

I'm trying to install NSX 6.4.0 in our test lab and when I'm trying to deploy the first NSX controller it fails in the vCenter Tasks and Events with the message: Operation timed out.

Selection_004.jpg

The log inside Networking & Security states the following:

Selection_005.jpg

And by looking at the log inside NSX Manager the first error in a long range of java exceptions is this one:

2018-05-02 16:04:46.479 CEST ERROR pool-45-thread-1 ResourcePoolVcOperationsImpl:196 - - [nsxv@6876 comp="nsx-manager" subcomp="manager"] Error while pushing file '/common/em/components/vdn/controller/ovf/nsx-controller-6.4.0-build7552024-system.vmdk'.

java.net.ConnectException: Connection timed out (Connection timed out)

        at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:1.8.0_151]

        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[?:1.8.0_151]

After this point there's a full range of java related garbage messages that continues for ~5 minutes before it gives up.

It seems that the error message Error while pushing file '/common/em/components/vdn/controller/ovf/nsx-controller-6.4.0-build7552024-system.vmdk'. is quite important here, but I haven't found any solutions yet. So far I've reviewed different kb articles (VMware Knowledge Base, VMware Knowledge Base) that used to be valid for 6.3.x, but it seems that the problem is different on 6.4.0.

DNS is setup correctly (VMware Knowledge Base ) and time is synced on all levels so I'm wondering if anyone has seen anything like this before. This is a totally clean environment that was just recently installed from scratch.

Lars

1 Solution

Accepted Solutions
larstr
Champion
Champion

The problem turned out to be firewall related. All is good now. Smiley Happy

View solution in original post

0 Kudos
10 Replies
mdac
Enthusiast
Enthusiast

Hi Lars,

Have you tried to deploy any other OVF templates from the vSphere Web Client to see if this is related to OVF deployment in general or something specific to the way NSX is deploying the controllers? I know that DNS on the ESXi hosts being incorrect can cause this type of problem, but I believe you've already got that covered.

You may also want to take a look at the /var/log/esxupdate.log file on the host the failed deployment occurred on. There may be some more helpful messaging there.

My blog: https://vswitchzero.com Follow me on Twitter: @vswitchzero
larstr
Champion
Champion

vswitchzero,

Thx for your input!

/var/log/esxupdate.log is not touched at all while trying to deploy the NSX-v controller. I tried deploying the vSAN witness appliance and it deployed completely within a minute or two.

Selection_324.jpg

Lars

0 Kudos
larstr
Champion
Champion

The problem turned out to be firewall related. All is good now. Smiley Happy

View solution in original post

0 Kudos
zwa88
Contributor
Contributor

Hi Lars,

We are also planning to set up a test lab with NSX. Did you actually get a cluster for setting up the lab?

Thanks!

0 Kudos
larstr
Champion
Champion

Yes, we're using a cluster with 3 hosts so we can also use vSAN in our lab.

Lars

0 Kudos
pelvis76
Contributor
Contributor

Hello

We have same issue in our plateform

Could you please provide detail regarding PORTS to be opened from NSX to vCenter ; as per matrix we need TCP 443 and 902 ; TCP 902 is not listening on vCenter only UDP 902 is in LISTENING STATE

Thanks for your support

0 Kudos
mdac
Enthusiast
Enthusiast

Hi pelvis76 - have a look at the following KB article that should give you a full list of required ports for NSX to various components. A quick glance through looks like TCP 443, 80 and 902 are required. You can use the 'debug connection' command from the NSX manager CLI to confirm these ports are open to the vCenter Server as well.

VMware Knowledge Base

Regards,

Mike

My blog: https://vswitchzero.com Follow me on Twitter: @vswitchzero
0 Kudos
iNik26
Enthusiast
Enthusiast

Hello Mike,

I'm having the same issue, controllers deployment fails with error:

"Pushing File

Operation failed on VC. For more details, refer to the rootCauseString or the VC logs"

I've no firewall between vCenter and NSX Manager but ports 902/903 seem to be closed on vCenter:

> debug connection 172.28.254.10

PING 172.28.254.10 (172.28.254.10): 56 data bytes

64 bytes from 172.28.254.10: icmp_seq=0 ttl=63 time=0.909 ms

64 bytes from 172.28.254.10: icmp_seq=1 ttl=63 time=0.960 ms

64 bytes from 172.28.254.10: icmp_seq=2 ttl=63 time=2.539 ms

--- 172.28.254.10 ping statistics ---

3 packets transmitted, 3 packets received, 0% packet loss

round-trip min/avg/max/stddev = 0.909/1.469/2.539/0.757 ms

172.28.254.10 reachable

172.28.254.10 reachable over port 80

172.28.254.10 reachable over port 443

172.28.254.10 not reachable over port 902

172.28.254.10 not reachable over port 903

Any ideas? Thank you,

kind regards,

Nicola

0 Kudos
mdac
Enthusiast
Enthusiast

Hi Nicola,

Sorry for the slow reply, I know I'm really late replying to this. Hopefully you already got it figured out, but just in case..

There are many different reasons that a controller deployment could fail with the 'Operation failed on VC' error. This holds true for ESGs, DLRs and controllers. The best thing to do would be to check the NSX manager logging after attempting a deployment to see more detailed information in the failure backtrace. The error message tells you to check the VC logging, but it's actually the NSX manager logging that will help. Common causes could be a datastore that was selected with insufficient space, or the lack of a 'compatible' host for the OVF image etc. The logging will have more information. You can see an example of one of these types of log messages in a troubleshooting post I did recently here:

https://vswitchzero.com/2018/12/07/nsx-troubleshooting-scenario-13-solution/

Regards,

Mike

My blog: https://vswitchzero.com Follow me on Twitter: @vswitchzero
0 Kudos
iNik26
Enthusiast
Enthusiast

Hi Mike,

yes, no worries and thanks for your reply. Yes, i've solved my issues. It was related to datastore "performances" . I've tried to use another one and i have been able to deploy all controllers.

Also I read your post, i often visit your blog .. very great job!

Thanks, regards,

Nicola