Just upgraded to 6.7 - All VMs show as disconnected and LDAP is not working

[aj800]

I was finally able to upgrade from VCSA 6.5U3 to 6.7U3 after correcting a STS issue resolved using this KB: VMware Knowledge Base

I ran into an issue where it failed due to NTP not being able to be configured, but I disabled NTP ion the VAMI and started over, which allowed the full upgrade to complete.

Once it was up, however, the browser's HTML5 loading UI kept spinning until I cleared the cache for it and reloaded the site.  I tried to login in using my LDAP account, but it didn't work.  I used my local account (and also the administrator account) and was able to get in.  Once I got in, I noticed all the VMs appeared to be off and show as "disconnected" in the vCenter list.  How do I fix this?  I learned this because I was looking to find out what happened with the old VCSA appliance and which host the new one is on.  Any assistance would help.  Thanks.

A bonus would be that when trying to reconfigure the NTP settings that were there prior to disabling them, I get a warning message that the IP is unreachable, and all this is after rebooting the VCSA 6.7 appliance.

[Gidrakos]

VMs (and their subsequent hosts) appearing offline likely means you need to disconnect and re-connect them (it wants you to re-enter the password for the host and/or re-initialize the cert). Verify the VMs are still running properly (they should be), then disconnect and re-connect to a host through the interface.

As for the login issues, double check that your ldap server is set as the default login and, if it is, you may need to remove and re-add it as an authentication source.

[msripada]

Are the hosts in sync or showing disconnected?

IF hosts showing disconnected the vms are disconnected which is expected.. We need to fix the host disconnection status

Ldap not working is that the authentication may not be working and need to vdalite network settings on VCSA like dns info etc

Thanks,

MS

[aj800 · Accepted solution]

So, there were a few issues we cleared up, with VMware support's assistance:

1. NTP and network issues - Our VCSA 6.5 instance was once configured on our production network subnet, and later moved to our management network by adding a second interface.  The network configuration on the VCSA showed that the VCSA was still using the production network's gateway and the Production network was still available.  There was strange routing configured and there were firewall permissions that allowed this.  This was all set up before I was managing this system.  The procedure for upgrading to 6.7 utilizes deployment of a new VCSA, but the deployment process only allows a single network interface of your choosing.  Since I chose the Management network interface to move to the new VCSA, and since the old one was still using the Production network and default gateway, it was not getting out to any other networks due to this configuration.  Once I changed the gateway to the Management network's gateway, NTP worked and it was able to at least reach the hosts but the VMs still showed as disconnected.

2. All VMs showed as "disconnected" after the upgrade completed - We were getting storage issues as reflected in the logs while disconnecting and reconnecting hosts to restore connectivity to the VMs.  VMware support had to go in and remove vDB entries since the local datastores of these hosts were already populated during the upgrade, and since vCenter was also trying to populate the db with the same information after the upgrade (reconnecting?) and failing since the entries were already there.  Once these entries were removed from the db, the hosts could be be added and the VMs appeared as active since the db was now able to populate with that information.

3. Joining new hosts failed with certificate issues - I was getting certificate issues when trying to join NEW hosts to a new host cluster in this datacenter in vSphere.  There is a vCenter setting (vCenter -> Configure -> Settings -> Advanced Settings -> vpxd.certmgmt.mode) with a default value of 'vmca', and VMware support had changed the value to 'thumbprint' which then allowed the new hosts to join the cluster using their default certificates (these were newly installed ESXi 6.7 hosts).  Once they were added successfully, this setting was changed back to its default 'vmca'.