hello,
to update this, VMware Support fixed the issue. There is pretty much no way I would have figured out how to do that. There were some issues with stale service registrations (whatever that means?), I believe because the certificates the services were registered with, were not synced or old...or something along those lines. The Rep ran a bunch of vmware cli command tools to get this fixed, I kind of got lost along the way of what he did. He also reset our administrator@vsphere.local password. After that patch completed successfully and the post install was fine, and it had the right permissions. I asked Customer Support Rep Manager to possibly have technical team put this in a kb, but I am not sure if that will be possible. For references and anyone else on the internet, this was the stuff that pointed us in the direction, which I saw in patchrunner.log...if you google-fu and came here, contact support 🙂
--------------
---------------
2022-06-06 10:30:00,654 - 33714 - update_b2b:: installRpms:2398 - INFO - Installation of all RPMs successfully completed
2022-06-06 10:30:00,654 - 33714 - dbfunctions:: executeDML: 54 - DEBUG - Executing Query {INSERT INTO progress VALUES(NULL, ?, DATETIME('now'), ?, ?, ?)} with parameters ('install', 'Installation of all RPMs successfully completed', 100, 70)
2022-06-06 10:30:00,656 - 33714 - task_manager:: update: 80 - DEBUG - UpdateTask: status=RUNNING, progress=80, message={'id': 'com.vmware.appliance.patch_in_progress', 'default_message': 'Converting data as part of post install', 'args': []}
2022-06-06 10:30:00,660 - 33714 - update_b2b:: _runPostInstall:1879 - DEBUG - Running the patch hook after rpm installation
2022-06-06 10:30:00,660 - 33714 - update_b2b:: _runScriptHook: 365 - DEBUG - Running B2B script with hook PatchHook
2022-06-06 10:30:00,660 - 33714 - update_b2b:: _runScriptHook: 374 - DEBUG - update script output to file /var/log/vmware/applmgmt/upgrade_hook_PatchHook
2022-06-06 10:30:00,663 - 33859 - script_executor:: executeHook: 41 - DEBUG - Add script folder /storage/updatemgr/software-updater_s09c6f/stage/scripts to python path
2022-06-06 10:30:00,673 - 33859 - extensions:: _findExtension: 83 - DEBUG - Found script hook <module 'update_script' from '/storage/updatemgr/software-updater_s09c6f/stage/scripts/update_script.py'>:PatchHook'
2022-06-06 10:30:00,673 - 33859 - extensions:: __call__: 105 - DEBUG - Executing script hook '<module 'update_script' from '/storage/updatemgr/software-updater_s09c6f/stage/scripts/update_script.py'>:PatchHook' with arguments <vmware.update.specs.UpdateSpec object at 0x7f2e040df050>.
2022-06-06 10:30:00,674 - 33859 - update_utils:: runCommand: 340 - DEBUG - Running command: ['/bin/bash', '--login', '-c', '/usr/bin/python3 "/storage/updatemgr/software-updater_s09c6f/stage/scripts/patches/PatchRunner.py" patch -u "/tmp/tmpapnr9394" -o "/var/log/vmware/applmgmt/upgrade_hook_PatchHook" -d "/storage/updatemgr/software-updater_s09c6f/stage/patch_runner" --disableStdoutLogging']
2022-06-06 10:30:00,675 - 33859 - update_utils:: runCommand: 345 - DEBUG - You can find the output of the command in temp files out /var/log/vmware/applmgmt/update-script-outbpk907co, err /var/log/vmware/applmgmt/update-script-errtyblomg1
2022-06-06 10:30:07,627 - 33859 - update_utils:: runCommand: 377 - DEBUG - STDOUT: b"2022-06-06T17:30:06.581Z Running command: ['/usr/lib/vmware-vmafd/bin/dir-cli', 'ssogroup', 'create', '--name', 'NsxAdministrators', '--description', 'SSO group to view and modify NSX configuration.', '--login', '']\n2022-06-06T17:30:06.604Z Done running command\n"
2022-06-06 10:30:07,628 - 33859 - update_utils:: runCommand: 378 - DEBUG - STDERR: b''
2022-06-06 10:30:07,628 - 33859 - update_script::_executePatchRunnerAndSupplyUserData: 125 - DEBUG - Patch command patch:
out=b"2022-06-06T17:30:06.581Z Running command: ['/usr/lib/vmware-vmafd/bin/dir-cli', 'ssogroup', 'create', '--name', 'NsxAdministrators', '--description', 'SSO group to view and modify NSX configuration.', '--login', ']\n2022-06-06T17:30:06.604Z Done running command\n"
error=b''
rc=1
2022-06-06 10:30:07,628 - 33859 - update_script::_executePatchRunnerAndSupplyUserData: 129 - ERROR - Patch command patch failed
2022-06-06 10:30:07,629 - 33859 - update_script:: _reportFailure: 161 - DEBUG - Error status has been already reported
2022-06-06 10:30:07,629 - 33859 - extensions:: __call__: 107 - DEBUG - The component script returned 'None'
2022-06-06 10:30:07,633 - 33714 - update_b2b:: _runScriptHook: 389 - DEBUG - update script result file '{\n "end_time": "2022-06-06T17:30:07.612Z",\n "error": {\n "componentKey": "wcp:Patch",\n "detail": [\n {\n "args": [\n "roles_groups_users",\n "{\\n \\"detail\\": [\\n {\\n \\"id\\": \\"install.ciscommon.command.errinvoke\\",\\n \\"translatable\\": \\"An error occurred while invoking external command : \'%(0)s\'\\",\\n \\"args\\": [\\n \\"Error 53 while creating SSO group \\\\\\"NsxAdministrators\\\\\\":\\\\ndir-cli failed. Error 53: Possible errors: \\\\nLDAP error: Server is unwilling to perform \\\\nWin Error: Operation failed with error ERROR_BAD_NETPATH (53) \\\\n\\"\\n ],\\n ..... [\n "Error 53 while creating SSO group \\"NsxAdministrators\\":\\ndir-cli failed. Error 53: Possible errors: \\nLDAP error: Server is unwilling to perform \\nWin Error: Operation failed with error ERROR_BAD_NETPATH (53) \\n"\n ],\n "localized": "An error occurred while invoking external command : \'Error 53 while creating SSO group \\"NsxAdministrators\\":\\ndir-cli failed. Error 53: Possible errors: \\nLDAP error: Server is unwilling to perform \\nWin Error: Operation failed with error ERROR_BAD_NETPATH (53) \\n\'"\n }\n ],\n "componentKey": null,\n "problemId": null,\n "resolution": null\n}']}, 'resolution': None}], 'info': [], 'warnings': []} from _runScriptHook
2022-06-06 10:30:07,633 - 33714 - update_b2b:: postInstallHook:2443 - DEBUG - PostInstall errors: [{'id': 'com.vmware.apliance.noid', 'time': '2022-06-06T17:30:07.633Z', 'message': {'id': 'wcp.patch.incrementalPatching.fail', 'default_message': 'Failed to apply patch roles_groups_users! Error: {\n "detail": [\n {\n "id": "install.ciscommon.command.errinvoke",\n "translatable": "An error occurred while invoking external command : \'%(0)s\'",\n "args": [\n "Error 53 while creating SSO group \\"NsxAdministrators\\":\\ndir-cli failed. Error 53: Possible errors: \\nLDAP error: Server is unwilling to perform \\nWin Error: Operation failed with error ERROR_BAD_NETPATH (53) \\n"\n ],\n "localized": "An error occurred while invoking external command : \'Error 53 while creating SSO group \\"NsxAdministrators\\":\\ndir-cli failed. Error 53: Possible errors: \\nLDAP error: Server is unwilling to perform \\nWin Error: Operation failed with error ERROR_BAD_NETPATH (53) \\n\'"\n }\n ],\n "componentKey": null,\n "problemId": null,\n "resolution": null\n}.', 'args': ['roles_groups_users', '{\n "detail": [\n {\n "id": "install.ciscommon.command.errinvoke",\n "translatable": "An error occurred while invoking external command : \'%(0)s\'",\n "args": [\n "Error 53 while creating SSO group \\"NsxAdministrators\\":\\ndir-cli failed. Error 53: Possible errors: \\nLDAP error: Server is unwilling to perform \\nWin Error: Operation failed with error ERROR_BAD_NETPATH (53) \\n"\n ],\n "localized": "An error occurred while invoking external command : \'Error 53 while creating SSO group \\"NsxAdministrators\\":\\ndir-cli failed. Error 53: Possible errors: \\nLDAP error: Server is unwilling to perform \\nWin Error: Operation failed with error ERROR_BAD_NETPATH (53) \\n\'"\n }\n ],\n "componentKey": null,\n "problemId": null,\n "resolution": null\n}']}, 'resolution': None}]
2022-06-06 10:30:07,634 - 33714 - task_manager:: update: 80 - DEBUG - UpdateTask: status=RESUMABLE, progress=80, message={'id': 'com.vmware.appliance.postinstall_errors', 'default_message': 'Data conversion/Post install hook failed', 'args': []}
2022-06-06 10:30:07,638 - 33714 - dbfunctions:: executeDML: 54 - DEBUG - Executing Query {INSERT INTO progress VALUES(NULL, ?, DATETIME('now'), ?, ?, ?)} with parameters ('failed', 'Installation failed', 100, 95)
2022-06-06 10:30:07,641 - 33714 - update_b2b:: postInstallHook:2450 - DEBUG - Returning from postInstallHook(), data conversion failed
2022-06-06 10:30:07,641 - 33714 - update_b2b:: postInstallHook:2457 - ERROR - Exception occured in postInstallHook
2022-06-06 10:30:07,641 - 33714 - task_manager:: update: 80 - DEBUG - UpdateTask: status=RESUMABLE, progress=80, message={'id': 'com.vmware.appliance.plain_message', 'default_message': '%s', 'args': ['Exception occured in postInstallHook']}
2022-06-06 10:30:07,644 - 33714 - update_b2b:: install:2684 - ERROR - Post-Install hook phase failed.
2022-06-06 10:30:07,645 - 33714 - dbfunctions:: executeDML: 54 - DEBUG - Executing Query {SELECT _value from config where _config = ?} with parameters ('failed-state',)
2022-06-06 10:30:07,645 - 33714 - dbfunctions:: executeDML: 54 - DEBUG - Executing Query {SELECT _value from config where _config = ?} with parameters ('failed-state',)
2022-06-06 10:30:07,645 - 33714 - dbfunctions:: executeDML: 54 - DEBUG - Executing Query {UPDATE config SET _value = ? where _config = ?} with parameters (40, 'failed-state')
2022-06-06 10:30:07,645 - 33714 - update_b2b:: install:2700 - DEBUG - Sending installation data to VAC
2022-06-06 10:30:07,645 - 33714 - dbfunctions:: openDb: 35 - INFO - Opening connection to database
2022-06-06 10:30:07,646 - 33714 - dbfunctions:: executeDML: 54 - DEBUG - Executing Query {SELECT _value from config where _config = ?} with parameters ('ceip_status',)
2022-06-06 10:30:07,646 - 33714 - dbfunctions:: executeDML: 54 - DEBUG - Executing Query {SELECT _value from config where _config = ?} with parameters ('toBuild',)
2022-06-06 10:30:07,646 - 33714 - dbfunctions:: executeDML: 54 - DEBUG - Executing Query {SELECT _value from config where _config = ?} with parameters ('fromBuild',)
2022-06-06 10:30:07,646 - 33714 - dbfunctions:: execute: 81 - DEBUG - Executing {SELECT * FROM meta}
2022-06-06 10:30:07,653 - 33714 - telemetry_functions:: send_to_vac: 76 - DEBUG - Sending data to 'http://localhost:15080/analytics/telemetry/ph/api/hyper/send/?_c=vCSA.7_0&_i=vcsa_patching'
2022-06-06 10:30:07,657 - 33714 - connectionpool:: _new_conn: 230 - DEBUG - Starting new HTTP connection (1): localhost:15080
2022-06-06 10:30:07,658 - 33714 - telemetry_functions:: send_to_vac: 87 - ERROR - Data is not sent, retrying to send the data, retry_count = 1/2
2022-06-06 10:30:07,659 - 33714 - connectionpool:: _new_conn: 230 - DEBUG - Starting new HTTP connection (1): localhost:15080
2022-06-06 10:30:07,660 - 33714 - telemetry_functions:: send_to_vac: 87 - ERROR - Data is not sent, retrying to send the data, retry_count = 2/2
2022-06-06 10:30:07,660 - 33714 - telemetry_functions:: telemetry: 125 - ERROR - Failed to send patching meta data to VAC server, Exception : {Retries are exhausted, giving in}
2022-06-06 10:30:07,661 - 33714 - update_microservice:: _runEvent: 391 - DEBUG - Event callback succeeded
2022-06-06 10:30:27,789 - 33714 - update_microservice:: waitForEvents: 455 - INFO - Exiting by timeout
2022-06-06 10:30:27,789 - 33714 - update_microservice:: __del__: 341 - DEBUG - Closing socket...
2022-06-06 10:30:27,789 - 33714 - update_microservice:: __del__: 343 - DEBUG - Removing sockfile
2022-06-06 10:30:27,789 - 33714 - update_microservice:: _deletePidFile: 279 - DEBUG - Removing pid file: /var/run/vmware/applmgmt/update_microservice.pid
-----------------------
