attempting to upgrade my lab from 6.7u3.latest to 7.0.latest
new VCSA VM deploys ok, but during pre-check get the following error:
Error
Resolution
I have already gone through the KB to no avail. I have also gone through and reset all certs (cert manager option 8).
Anyone have any guidance or suggestions?
Thanks,
-GB
Hi gjbrown,
I have attached a script here.
Please download the script and run it on the source machine to fix any ssl trust mismatch in lookup service registrations.
Please take a snapshot before proceeding.
Copy the file to lstool scripts folder.
For vCSA path:
# /usr/lib/vmidentity/tools/scripts
Run the below commands:
# python ls_ssltrust_fixer.py -f scan
#python ls_ssltrust_fixer.py -f fix
Then try running the upgrade.
Note: Make sure you take necessary backup/snapshot. Please try this ls_ssltrust_fixer.py in test environment, do not try this in production environment. Please raise a support request to validate before executing this script in production environment.
Regards,
Sudeshna Sarkar
Install-Upgrade Specialist
_______________________________________________________________________________________________________
"Did you find this helpful? Let us know by completing this survey (takes 1 minute!)"
thanks for the suggestion, but tried that as well.
GB
Moderator: Thread moved to the vSphere Upgrade & Install area.
This issue mostly occurs if the SSL trust of the services registered on PSC are having different than the SSL certificate of the node (of which the services is registered).
Please follow steps of the below article
you have to basically get the old thumbprint and update the services with ls update cert script using the new SSL certificate which is currently present
This command will give you all the services registered along with SSL trust they have .
/usr/lib/vmidentity/tools/scripts/lstool.py list --url https://localhost/lookupservice/sdk --no-check-cert --ep-type com.vmware.cis.cs.identity.sso 2>/dev/null
*Please mark the answer as correct if it solves your query
Thanks harry89, I went through the KB no errors, replaced 3 certificates but still the same issue when I attempt to upgrade.
-GB
Can u send the log snippet
harry89 which log snip you want? the log bundle compressed is 16mb and I am sure you don't want to deal with all of it.
Thx
Hi gjbrown,
You can run the following command to check if the certificates of the existing environment is fine and valid or not .
#for i in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list); do echo $i; /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $i --text | grep -i "not after"; done;
If the certs are fine and you continue to face the same issue please go ahead and replace the certificates using the option 8 in the certificate-manager tool.
Then continue with the upgrade again.
It still you run into any issue please open a support request with us.
Regards,
Sudeshna Sarkar
Install-Upgrade Specialist
Hi sudeshnas
When I ran the command you provided it only returned back to a prompt with no output. Not sure if that is good or bad.
I ran through cert replacement, option 8 again, even though I have done already.
Updated 5 service(s)
Status : 60% Completed [Reset vpxd-extension Cert...]
2020-07-22T15:14:46.910Z Updating certificate for "com.vmware.imagebuilder" extension
Reset status : 100% Completed [Reset completed successfully]
--obviously this is good.
but upgrade still fails
Hi gjbrown,
I have attached a script here.
Please download the script and run it on the source machine to fix any ssl trust mismatch in lookup service registrations.
Please take a snapshot before proceeding.
Copy the file to lstool scripts folder.
For vCSA path:
# /usr/lib/vmidentity/tools/scripts
Run the below commands:
# python ls_ssltrust_fixer.py -f scan
#python ls_ssltrust_fixer.py -f fix
Then try running the upgrade.
Note: Make sure you take necessary backup/snapshot. Please try this ls_ssltrust_fixer.py in test environment, do not try this in production environment. Please raise a support request to validate before executing this script in production environment.
Regards,
Sudeshna Sarkar
Install-Upgrade Specialist
_______________________________________________________________________________________________________
"Did you find this helpful? Let us know by completing this survey (takes 1 minute!)"
There are possibilities that when u ran the reset all the certificates , some of the endpoints are still having the older machine SSL cert as ssl trust .
This is fairly common occurrence .
But was this done before starting the upgrade or after . (reset all certificates).
If this was done to try to mitigate the issue and solve the upgrade problem , then not sure if this right direction because we need to be sure that prior to upgrade some cert in vecs-cli was surely expired and that was machine ssl .
Hi sudeshnas
The script worked and found 31 mismatches. I ran the fix which let me run the upgrade but failed @ error#2, 89%. Here is the error
Error
WCP service installation failed : Traceback (most recent call last): File "/usr/lib/vmware-wcp/firstboot/wcp-firstboot.py", line 50, in proxy return func(*args, **kwargs) File "/usr/lib/vmware-wcp/firstboot/wcp-firstboot.py", line 71, in configure wcpconfigure.configure_service() File "/usr/lib/vmware-wcp/py-modules/wcpconfigure.py", line 442, in configure_service create_storage_identity() File "/usr/lib/vmware-wcp/py-modules/wcpconfigure.py", line 438, in create_storage_identity SsoUser(_STORAGE_USER).create_storage_user_and_assign() File "/usr/lib/vmware-wcp/py-modules/wcpconfigure.py", line 330, in create_storage_user_and_assign self._create_storage_user() File "/usr/lib/vmware-wcp/py-modules/wcpconfigure.py", line 298, in _create_storage_user password = svcacctmgmt_client.create_svc_account(self._user_name) File "/usr/lib/vmware-wcp/py-modules/svcacctmgmt.py", line 90, in create_svc_account raise er File "/usr/lib/vmware-wcp/py-modules/svcacctmgmt.py", line 84, in create_svc_account svcacct_pwd_out = svcacct_client.create(create_spec) File "/usr/lib/vmware-wcp/py-modules/vapi-bindings/com/vmware/vcenter/svcaccountmgmt_client.py", line 368, in create 'create_spec': create_spec, File "/usr/lib/vmware-vapi/lib/python/vapi_runtime-2.100.0-py2.py3-none-any.whl/vmware/vapi/bindings/stub.py", line 345, in _invoke return self._api_interface.native_invoke(ctx, _method_name, kwargs) File "/usr/lib/vmware-vapi/lib/python/vapi_runtime-2.100.0-py2.py3-none-any.whl/vmware/vapi/bindings/stub.py", line 298, in native_invoke self._rest_converter_mode) com.vmware.vapi.std.errors_client.InternalServerError: {messages : [LocalizableMessage(id='com.vmware.vapi.authorization.permission.error', default_message='Could not validate permission information for operation com.vmware.vcenter.svcaccountmgmt.service_account.create invocation.', args=['com.vmware.vcenter.svcaccountmgmt.service_account.create'], params=None, localized=None)], data : None, error_type : None}
Resolution
This is an unrecoverable error, please retry install. If you encounter this error again, please search for these symptoms in the VMware Knowledge Base for any known issues and possible resolutions. If none can be found, collect a support bundle and open a support request.
I do have SR 20142056507 open, but just getting started if you would like to review any logs.
Thank you for the help with this.
Hi gjbrown,
Thank you for opening a ticket with us.
I have gone through the logs and the errors/backtrace reported.
Well upon researching I see that similar issue has been reported by the other customer too and currently we are working internally to get it fixed.
You will receive all the updates on the ticket.
Regards,
Sudeshna Sarkar
Install-Upgrade Specialist
sudeshnas Thanks for digging into this. I'll see what GSS says via ticket. I'll update this thread with info to guide others towards a KB or solution.
Again thanks for the help and time with this.
-GB
sudeshnas, that script worked perfectly for me, thank you!
I had some invalid cert, that not even regenerating and resetting existing certs worked to resolve.
Gave your script a shot, and bam!
Perfect, worked for me - Thanks
Hi,
I have the same error : WCP service installation failed.
Where can i find the solution for this problem ?
Thanks for help.
Hello,
your script returns the following error:
root@vcenter [ /usr/lib/vmidentity/tools/scripts ]# python ls_ssltrust_fixer_p3.py -f fix
Running function 'fix'
Fix phase 1: Reading IDs with incorrect certificate from scan results
Using mismatch ID list from: /var/log/ls_ssltrust_fixer/mismatchIDs
SSO administrator user (Default:Administrator@vsphere.local):administrator@vsphere.local
Traceback (most recent call last):
File "ls_ssltrust_fixer_p3.py", line 368, in <module>
main()
File "ls_ssltrust_fixer_p3.py", line 360, in main
_doFix()
File "ls_ssltrust_fixer_p3.py", line 297, in _doFix
user=input("SSO administrator user (Default:Administrator@vsphere.local):") or "Administrator@vsphere.local"
File "<string>", line 1
administrator@vsphere.local
^
SyntaxError: invalid syntax
I ran into the same issue.. Adding " " around the user name worked for me. i.e. "administrator@vsphere.local"