Upgrade 6.7 Windows 2016 VCenter to VCenter 8.0U1a

[Tony202200]

Hello comrades,

I have been trying to upgrade my vmware environment from 6.7u3s to 8.0u1a. 

All my 6 attempts have failed at the same spot in stage 2 of starting the STS service. Please see attached.

My environment Windows AD:

1x flat network: 192.168.0.0/24. No firewall and all devices are able to ping each other (incl. VMs)

2x 7.0 esxi hosts (joined to AD)

1x 6.7 VCenter (Windows 2016) added to AD, forward & reverse DNS setup and resolvable

I am aware that I need to perform a migration to vmware 8 as opposed to upgrade. 

The initial SSO admin was the default: administator@vsphere.local. I used it on a few attempts which failed on stage 2 starting STS. Some research alluded to using AD SSO instead.

I then added my AD domain administrator account as another SSO (set it as default). This new account was added to VCenter 6.7's user and group management and gave it an Administrative role with global permissions. As a test I was able to sign onto VCenter 6.7 and get access to the objects (Vcenter, DataCenter, Clusters etc...). Two more attempts were done again ending with exactly the same problem defined above. 

Researching again, I came across suggestions to ensure DNS lookups are working (which they are in my case as defined earlier). 

The temporary IP that I needed to have was available in my environment. During the migration VC6.7 shuts down as expected. 

After the failures, I sign onto the ESXi host and power down the broken VC8 which does get created and bring up VC6.7. The VC8.0 VM/Appliance keeps VC6.7's FQDN but using the temporary IP (which I guess never releases because the migration fails).

I hope I am not deluding myself into accepting that VC8.0 can be hosted on ESXi7.0. I have VC6.7 running happily on the same host as I am trying to deploy VC8.0 to.

I am at a loss at this stage. 

Appreciate your guidance!

Thanks,

Tony

[markey165]

Random suggestion, have you tried exiting the upgrade at Stage 1, then continuing Stage 2 directly from the VAMI interface?

I've never needed to do it myself, but i've read that support have suggested this for some upgrade failure scenarios. It may not be the solution in your case, but simple enough to try. Failing that i'd be raising a support call with GSS, they should be able to assist 

HTH

[Kinnison]

Hello,

Just an observation worth what it's worth, if there are no other particular reasons if I had to perform an upgrade I would use the most recent version of the vCenter object currently available, 8.0U1c

Regards,
Ferdinando

[Ajay1988]

Please check https://kb.vmware.com/s/article/2148950

[schikhaldekar]

I believe the problem is not with target version but with existing VC 6.7 somewhere.

Could you please check below:

- STS certificate : https://kb.vmware.com/s/article/79248

- Verify all certificates of vCenter: https://kb.vmware.com/s/article/82332

- Verifying upgrade logs : https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-vcenter-installation/GUID-0F24A762-F7AE-4B8F-A...

The STS failure log attached seems generic. 

[Acebandge]

Your not deluding your self.  VC8 will run on a ESXI7 Host.  I'm running mine like that.  Once you have it up and running and your 7.0 hosts are joined to VC8 it will give you an annoying Warning message across the top bar telling you that you are running a "Mixed environment", but I just ignore it, cause I'm not made of money.  And I can't go buy a new server every time VMware want's to remove perfectly good hardware from their compatibility list.  But anyway that's a different post. 

As far as the DNS, you have to manually add the DNS entries into the Domain Controller, by running the DNS Manager, browse to Forward Lookup Zone, right click, create a new Host A record, then add the same in the reverse lookup zone.  Joining the network domain won't create the DNS entries, which causes you to browse by IP and not by the VC server's FQDN.

I had to read your post again to make sure I understood.  It sounds like your trying to upgrade a Windows 2016 VC to a VC8 virtual appliance, but I don't think that will work.  I know the older versions would convert but it sounds like the hard way.  I may be wrong.

But.. just in case, Sorry if this is too basic. 

Instead of upgrading 6.7, I would just setup a clean VC8.  Download the VC8 ISO right click and "mount" it on your computer and run the VCSA 64bit installer.  And point it to your 7.0 host box, using it's root account and PW (Not Administrator or Domain account).  Go through the 2 phase process, then you'll have a nice clean VC8 appliance.  Then you can disconnect your ESXI 7.0 boxes from the old 6.7 VC.  Then login to VC8 and connect them to it and start rejoining your domain again. 

If this fails, you may want to disjoin one of your ESXI7.0 boxes from the domain and from the VC6.7 so that it is a stand alone host. Then try the VC8 setup with root account again. 

One thing that could be causing the failure in the upgrade process could be the upgrade is attempting to recreate the resources, Ram CPU, Drive space (with partitions) that the 6.7 server is using.  But the 7.0 host doesn't have enough.  I've seen that break the upgrade process. 

[Tony202200]

Invaluable suggestions Ace... much appreciated.

I have done just that (i.e, standup a nice clean VC8) before reading your post. I needed to make sure it wasn't a corrupt iso or something weird along those lines. Turns out that VC8 installed without a glitch without performing a migration.

I needed to do a migration so as to get all the metrics, configuration etc... across from VC6.7 to VC8. 

[Tony202200]

Hi Schikhaldekar,

Many thanks for your reply.

I tried to follow your guidance for checking the STS certificate; however clicking on the login and manage certificates button is unresponsive. I have used Edge external to the VC67 server and IE11 internal to the server.

Is it an activex/java problem?

As for logs, please see attached.

[schikhaldekar]

@Tony202200 ,

Thank you for uploading log details. Still log doesn't show much information about why exactly upgrade is failing.

2023-09-13T03:10:40.424Z - debug: Open url in external window from VAMI: [object Event]
2023-09-13T03:10:40.460Z - info: ERROR Error: Uncaught (in promise): Error: Cannot match any routes. URL Segment: 'installer'
Error: Cannot match any routes. URL Segment: 'installer'
    at P.value (https://192.168.0.36:5480/main.d9a53162f248acdb.js:1:575008)
    at https://192.168.0.36:5480/main.d9a53162f248acdb.js:1:574666
    at https://192.168.0.36:5480/main.d9a53162f248acdb.js:1:264390
    at fe._error (https://192.168.0.36:5480/main.d9a53162f248acdb.js:1:263616)
    at fe.value (https://192.168.0.36:5480/main.d9a53162f248acdb.js:1:243530)
    at fe.value [as _error] (https://192.168.0.36:5480/main.d9a53162f248acdb.js:1:243912)
    at fe.value (https://192.168.0.36:5480/main.d9a53162f248acdb.js:1:243530)
    at fe.value [as _error] (https://192.168.0.36:5480/main.d9a53162f248acdb.js:1:243912)
    at fe.value (https://192.168.0.36:5480/main.d9a53162f248acdb.js:1:243530)
    at fe.value [as _error] (https://192.168.0.36:5480/main.d9a53162f248acdb.js:1:243912)

I checked with above failure message and it can be due to multiple issues on source vCenter itself.

- PSC issue

- DNS forward and reserve lookup for existing and new IP to FQDN

- vmdird services which is in read only status

- From STS certificate KB, There are ways to check for certificate status with script too. Please do verify the same.

- Apart from uploaded migration logs, We need to check other service logs as well to figure out which service is failing to respond to migration cordinator which interns fail.

- On source VC and target VC, During the time of upgrade you need to check multiple set of logs to figure out the problem.

- At the moment, Best bet is to engage VMware GSS as they have amazing scripts which can figure out VC problem in no time.

[Tony202200]

Hi Ace,

I hope you don't mind my calling you this :). 

After a clean install of VC8, the server worked ok until I had to restart it. Post restart most of the services are failing to restart from checking VAMI. 

I am back to the drawing board again with the STS service is 'starting' state and not starting. Other services are perhaps depending on this service to start.

Did you by any chance run into a similar issue when you installed your VC8 instance in a clean state (i.e., not migrating or upgrading)?

Thanks again,

Tony