NSX-T REPO SYNC FAILED

[ggovek]

Hi,

I have 3 NSX-T Managers and I have a  repo sync problem on two of them. How to resolve this problem?

I try to reboot all three appliances and this is not help. Also I try with Resolve button.

[virtualgeoff]

I have this problem too.

Did you find a fix for this issue?

[jonedved]

Hi,

I have the same issue. Running 4.1.0.0 by the way.

Searching KB, internet - no hits whatsoever. 

Does anyone know how to troubleshoot this issue? Which logs and services to check? 

I had a look through bunch of them - but I can't get this to work. One strange thing I noticed - it seems NSX database has a fqdn saved in the IPv4 field of NSX Manager nodes under get nodes verbose:

UUID : xx
Label : Manager
IPv4 Addresses : ['nsx-manager3.lab.local']
IPv6 Addresses : N/A
Hostname/FQDN : nsx-manager3.lab.local
Display Name : nsx-manager3

UUID : xx
Label : Manager
IPv4 Addresses : ['nsx-manager2.lab.local']
IPv6 Addresses : N/A
Hostname/FQDN : nsx-manager2.lab.local
Display Name : nsx-manager2

UUID : xx
Label : Manager
IPv4 Addresses : ['nsx-manager1.lab.local']
IPv6 Addresses : N/A
Hostname/FQDN : nsx-manager1.lab.local
Display Name : nsx-manager1.lab.local

There is a reference to invalid IP in proton logs:

2023-06-02T13:51:36.516Z ERROR http-nio-127.0.0.1-7440-exec-3 IPv4Address 4407 - [nsx@6876 comp="nsx-manager" errorCode="MP260" level="ERROR" reqId="d385ced2-52e5-4be0-808e-049e646a00df" subcomp="manager" username="uproton"] Invalid ip string 'nsx-manager1.lab.local'. Error parsing 'nsx-manager1'.

Also - there is a reference to missing IP: 

2023-06-02T15:09:44.833Z WARN http-nio-127.0.0.1-7440-exec-1 FacadeInterceptorHelperImpl 4452 - [nsx@6876 comp="nsx-manager" level="WARNING" reqId="df7a2453-7ffd-45c1-addd-6b4644d994c0" subcomp="manager" username="UC"] API result did not pass validationField level validation errors: {required property results[3].controller_role.control_cluster_listen_addr.ip_address is missing}, {required property results[4].controller_role.control_cluster_listen_addr.ip_address is missing}

[avaharagov]

Good afternoon, encountered the same problem on NSX 4.1.0, could you please tell me if you were able to solve this problem?

[mynameisalex]

I have the same problem. Is there any solution to the problem?

[jonedved]

I got it fixed - worked on it together with VMware support.

It seems there are potentially two problems:

1./ FQDN in IPv4Address field on NSX Manager "get node" output. If this happens - I suspect RepoSync process can't work.

2./ RepoSync failing for another reason - this reason can be discovered in nsxapi.log.

Login to NSX Manager CLI as admin user - then check that you have FQDN in IPv4Address field using "get node" command?

[avaharagov]

There is no FQDN in the IPv4Address field. There are errors in nsxapi.log that clearly show that the repository is going to synchronize files that are not there

[jonedved]

The question is why? 

The correct way to fix it would be engaging support. 

I have manually uploaded the bundle to my repository server. This however is not supported - while I can't see anything wrong with it, support will probably not like it. 

Example - RepoSync is complaining about missing files for "4.1.0.0.0.21332672" version:

I downloaded the 4.1.0.0 MUB (upgrade bundle), then extracted all the files with 7z (gunzip should work). Then copied the entire extracted package to my repository node /repository/4.1.0.0.0.21332672/ with scp.

Last step was to change owner on the files I copied - in this example: chown -R uuc:grepodir 4.1.0.0.0.21332672\

After that, issue a manual repo-sync with API and it should make progress. 

Let us know if that helps.

[tayyabnono]

@jonedved 

I'm facing the REPO_SYNC issue, and I have tried all the mentioned methods and now I don't know how to solve it currently I'm running version  4.1.0.2.0.21761695. I have ALB running on Manager.