Solved: Re: external PSC 6.0 upgrade to 6.5 fails - Intern...

andreir · ‎09-26-2018

Hi all,

trying to upgrade external PSC from 6.0.0.20000 to 6.5 U2, the new appliance is deployed, but then the upgrade process fails during the pre-upgrade check.

using cli-installer:

Precheck error: 1. Title: Internal error occurs during pre-upgrade checks.

Precheck error: 1. Suggested Resolution: Please search for these symptoms in the

VMware Knowledge Base for any known issues and possible resolutions. If none can

be found, collect a support bundle and open a support request.

Failed normalizing ip: psc04.vlab.local

(u'Failed to uninstall %(rpm)s on %(host)s: %(error)s', {'host':

u'psc04.vlab.local', 'rpm': u'vmware-upgrade-requirements-*', 'error':

CommandError(u"Failed to run and wait for command in guest with error 'Command

'/bin/rpm' exited with non-zero status 1'",)})

Collecting support bundle from source VM psc04.vlab.local ...

Failed normalizing ip: psc04.vlab.local

Collecting the support bundle from the deployed appliance...

Got error while running OVF Tool command: Errors encountered during the precheck

process. Correct the errors and provide a new appliance name in the template,

because you must deploy a new appliance.

using gui-installer:

Checked /var/log/vmware/upgrade on the source and see a bunch of sso errors and not a lot of other clues. Any idea how to overcome this issue?

/var/log/vmware/upgrade/requirements-upgrade-runner.log

...

2018-09-26T17:41:48.945Z ERROR bindings.sso_binding Cannot log-in to SSO admin service, because of Error: (vmodl.fault.InvalidRequest) {

dynamicType = <unset>,

dynamicProperty = (vmodl.DynamicProperty) [],

msg = "Request version 'urn:sso/version3_5' and namespace 'urn:sso' are not supported",

faultCause = <unset>,

faultMessage = (vmodl.LocalizableMessage) []

}.

Traceback (most recent call last):

File "/var/tmp/vmware-upgrade-requirements/cis-upgrade-runner/py/bindings/sso_binding.py", line 138, in createSession

content = si.RetrieveServiceContent()

File "/var/tmp/vmware-upgrade-requirements/cis-upgrade-runner/libs/pyVmomi.zip/pyVmomi/VmomiSupport.py", line 557, in <lambda>

self.f(*(self.args + (obj,) + args), **kwargs)

File "/var/tmp/vmware-upgrade-requirements/cis-upgrade-runner/libs/pyVmomi.zip/pyVmomi/VmomiSupport.py", line 363, in _InvokeMethod

return self._stub.InvokeMethod(self, info, args)

File "/var/tmp/vmware-upgrade-requirements/cis-upgrade-runner/libs/pyVmomi.zip/pyVmomi/SoapAdapter.py", line 1338, in InvokeMethod

raise obj # pylint: disable-msg=E0702

vmodl.fault.InvalidRequest: (vmodl.fault.InvalidRequest) {

dynamicType = <unset>,

dynamicProperty = (vmodl.DynamicProperty) [],

msg = "Request version 'urn:sso/version3_5' and namespace 'urn:sso' are not supported",

faultCause = <unset>,

faultMessage = (vmodl.LocalizableMessage) []

}

2018-09-26T17:41:48.948Z WARNING bindings.sso_binding Could not retrieve service instance from SSO component using API version 'sso.version.version3_5'. Falling back to API version 'sso.version.version1'.

...

Other errors and warnings from requirements-upgrade-runner.log that might be relevant.

psc04:/var/log/vmware/upgrade # grep -iHE "error|warning" *

requirements-upgrade-runner.log:2018-09-26T17:39:33.286Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:39:33.287Z ERROR utils.featureState_utils Can not get AUTOMATED_VUM_UPGRADE feature state, default will be used

requirements-upgrade-runner.log:ImportError: No module named featureState

requirements-upgrade-runner.log:2018-09-26T17:39:33.302Z ERROR bindings.sso_binding Cannot log-in to SSO admin service, because of Error: (vmodl.fault.InvalidRequest) {

requirements-upgrade-runner.log:2018-09-26T17:39:33.305Z WARNING bindings.sso_binding Could not retrieve service instance from SSO component using API version 'sso.version.version3_5'. Falling back to API version 'sso.version.version1'.

requirements-upgrade-runner.log:2018-09-26T17:39:33.845Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:39:33.860Z ERROR bindings.sso_binding Cannot log-in to SSO admin service, because of Error: (vmodl.fault.InvalidRequest) {

requirements-upgrade-runner.log:2018-09-26T17:39:33.861Z WARNING bindings.sso_binding Could not retrieve service instance from SSO component using API version 'sso.version.version3_5'. Falling back to API version 'sso.version.version1'.

requirements-upgrade-runner.log:2018-09-26T17:39:33.948Z WARNING model.components_metadata Cannot find matching source metadata for provided:platform=Linux, version=6.0 and source-metadata=[{u'export-after': [u'vpxd'], u'platform': [u'Windows'], u'version': [u'5.5'], u'export-from': [u'com.vmware.vpxd']}]

requirements-upgrade-runner.log:2018-09-26T17:39:33.948Z WARNING model.components_metadata Invalid metadata: {u'destination-system': [{u'upgrade-script': u'vco', u'description': u'VMware vCenter Orchestrator component', u'component-display-name': u'VMware vCenter Orchestrator', u'platform': [u'Windows', u'Linux'], u'version': u'6.1', u'component-name': u'vco', u'optional': True}], u'source-system': [{u'export-after': [u'vpxd'], u'platform': [u'Windows'], u'version': [u'5.5'], u'export-from': [u'com.vmware.vpxd']}]}

requirements-upgrade-runner.log:2018-09-26T17:39:33.950Z WARNING model.components_graph Component com.vmware.sso depends on follow components set([u'com.vmware.common_logging', u'com.vmware.rdb']) but the system does not have metadata for them.

requirements-upgrade-runner.log:2018-09-26T17:39:33.951Z WARNING model.components_graph Component com.vmware.vmafd depends on follow components set([u'com.vmware.common_logging', u'com.vmware.rdb']) but the system does not have metadata for them.

requirements-upgrade-runner.log:2018-09-26T17:39:33.958Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:39:33.959Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:39:33.959Z WARNING model.components Component com.vmware.rbd won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:39:33.959Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:39:33.960Z WARNING model.components Component com.vmware.vpxd won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:39:33.960Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:39:33.960Z WARNING model.components Component com.vmware.vcdb won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:39:33.960Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:39:33.961Z WARNING model.components Component com.vmware.eam won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:39:33.961Z WARNING model.components Component com.vmware.ngc won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:39:33.961Z WARNING model.components Component com.vmware.netdump won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:39:33.962Z WARNING model.components Component com.vmware.is won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:39:33.962Z WARNING model.components Component com.vmware.vsan-health won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:39:33.962Z WARNING model.components Component com.vmware.vcIntegrity won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:39:33.962Z WARNING model.components Component com.vmware.sps won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:39:33.963Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:39:33.964Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:39:33.964Z WARNING model.components Component com.vmware.cls won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:39:33.977Z ERROR bindings.sso_binding Cannot log-in to SSO admin service, because of Error: (vmodl.fault.InvalidRequest) {

requirements-upgrade-runner.log:2018-09-26T17:39:33.978Z WARNING bindings.sso_binding Could not retrieve service instance from SSO component using API version 'sso.version.version3_5'. Falling back to API version 'sso.version.version1'.

requirements-upgrade-runner.log:2018-09-26T17:39:34.43Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:39:52.466Z WARNING upgrade.states.preupgrade_state vpxd_servicecfg does not exist, assume local storage

requirements-upgrade-runner.log:2018-09-26T17:39:52.638Z WARNING deployer.migration_env_deployer Could not unregister component com.vmware.migrate-connector.127.0.0.1 located on localhost as extension. Exception: -- 'NoneType' object has no attribute 'content'

requirements-upgrade-runner.log:2018-09-26T17:41:33.413Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:41:33.414Z ERROR utils.featureState_utils Can not get AUTOMATED_VUM_UPGRADE feature state, default will be used

requirements-upgrade-runner.log:ImportError: No module named featureState

requirements-upgrade-runner.log:2018-09-26T17:41:33.432Z ERROR bindings.sso_binding Cannot log-in to SSO admin service, because of Error: (vmodl.fault.InvalidRequest) {

requirements-upgrade-runner.log:2018-09-26T17:41:33.435Z WARNING bindings.sso_binding Could not retrieve service instance from SSO component using API version 'sso.version.version3_5'. Falling back to API version 'sso.version.version1'.

requirements-upgrade-runner.log:2018-09-26T17:41:33.685Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:41:33.703Z ERROR bindings.sso_binding Cannot log-in to SSO admin service, because of Error: (vmodl.fault.InvalidRequest) {

requirements-upgrade-runner.log:2018-09-26T17:41:33.705Z WARNING bindings.sso_binding Could not retrieve service instance from SSO component using API version 'sso.version.version3_5'. Falling back to API version 'sso.version.version1'.

requirements-upgrade-runner.log:2018-09-26T17:41:33.797Z WARNING model.components_metadata Cannot find matching source metadata for provided:platform=Linux, version=6.0 and source-metadata=[{u'export-after': [u'vpxd'], u'platform': [u'Windows'], u'version': [u'5.5'], u'export-from': [u'com.vmware.vpxd']}]

requirements-upgrade-runner.log:2018-09-26T17:41:33.798Z WARNING model.components_metadata Invalid metadata: {u'destination-system': [{u'upgrade-script': u'vco', u'description': u'VMware vCenter Orchestrator component', u'component-display-name': u'VMware vCenter Orchestrator', u'platform': [u'Windows', u'Linux'], u'version': u'6.1', u'component-name': u'vco', u'optional': True}], u'source-system': [{u'export-after': [u'vpxd'], u'platform': [u'Windows'], u'version': [u'5.5'], u'export-from': [u'com.vmware.vpxd']}]}

requirements-upgrade-runner.log:2018-09-26T17:41:33.799Z WARNING model.components_graph Component com.vmware.sso depends on follow components set([u'com.vmware.common_logging', u'com.vmware.rdb']) but the system does not have metadata for them.

requirements-upgrade-runner.log:2018-09-26T17:41:33.800Z WARNING model.components_graph Component com.vmware.vmafd depends on follow components set([u'com.vmware.common_logging', u'com.vmware.rdb']) but the system does not have metadata for them.

requirements-upgrade-runner.log:2018-09-26T17:41:33.807Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:41:33.808Z WARNING model.components Component com.vmware.rbd won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:41:33.808Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:41:33.808Z WARNING model.components Component com.vmware.vpxd won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:41:33.809Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:41:33.809Z WARNING model.components Component com.vmware.vcdb won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:41:33.809Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:41:33.809Z WARNING model.components Component com.vmware.eam won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:41:33.810Z WARNING model.components Component com.vmware.ngc won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:41:33.810Z WARNING model.components Component com.vmware.netdump won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:41:33.810Z WARNING model.components Component com.vmware.is won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:41:33.810Z WARNING model.components Component com.vmware.vsan-health won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:41:33.811Z WARNING model.components Component com.vmware.vcIntegrity won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:41:33.811Z WARNING model.components Component com.vmware.sps won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:41:33.811Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:41:33.812Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:41:33.812Z WARNING model.components Component com.vmware.cls won't be upgraded

requirements-upgrade-runner.log:2018-09-26T17:41:33.825Z ERROR bindings.sso_binding Cannot log-in to SSO admin service, because of Error: (vmodl.fault.InvalidRequest) {

requirements-upgrade-runner.log:2018-09-26T17:41:33.826Z WARNING bindings.sso_binding Could not retrieve service instance from SSO component using API version 'sso.version.version3_5'. Falling back to API version 'sso.version.version1'.

requirements-upgrade-runner.log:2018-09-26T17:41:33.888Z WARNING networking_utils Could not find address info for 127.0.0.1

requirements-upgrade-runner.log:2018-09-26T17:41:47.190Z WARNING upgrade.states.preupgrade_state vpxd_servicecfg does not exist, assume local storage

requirements-upgrade-runner.log:2018-09-26T17:41:47.356Z WARNING deployer.migration_env_deployer Could not unregister component com.vmware.migrate-connector.127.0.0.1 located on localhost as extension. Exception: -- 'NoneType' object has no attribute 'content'

upgrade-linux-requirements.log:2018-09-26T17:39:30.649Z ERROR utils.featureState_utils Can not get AUTOMATED_VUM_UPGRADE feature state, default will be used

upgrade-linux-requirements.log:ImportError: No module named featureState

upgrade-linux-requirements.log:2018-09-26T17:39:54.513Z ERROR bindings.sso_binding Cannot log-in to SSO admin service, because of Error: (vmodl.fault.InvalidRequest) {

upgrade-linux-requirements.log:2018-09-26T17:39:54.527Z WARNING bindings.sso_binding Could not retrieve service instance from SSO component using API version 'sso.version.version3_5'. Falling back to API version 'sso.version.version1'.

upgrade-linux-requirements.log:2018-09-26T17:41:31.706Z ERROR utils.featureState_utils Can not get AUTOMATED_VUM_UPGRADE feature state, default will be used

upgrade-linux-requirements.log:ImportError: No module named featureState

upgrade-linux-requirements.log:2018-09-26T17:41:48.945Z ERROR bindings.sso_binding Cannot log-in to SSO admin service, because of Error: (vmodl.fault.InvalidRequest) {

upgrade-linux-requirements.log:2018-09-26T17:41:48.948Z WARNING bindings.sso_binding Could not retrieve service instance from SSO component using API version 'sso.version.version3_5'. Falling back to API version 'sso.version.version1'.

RickVerstegen · ‎09-26-2018

ERROR bindings.sso_binding Cannot log-in to SSO admin service, because of Error: (vmodl.fault.InvalidRequest) {

dynamicType = <unset>,

Looks like a bad SSO password regarding the logs.

Change the SSO password to a supported password.and restart the upgrade process.

_______________________________________________________________________________________________________

"Did you find this helpful? Let us know by completing this survey (takes 1 minute!)"

Was I helpful? Give a kudo for appreciation!
Blog: https://rickverstegen84.wordpress.com/
Twitter: https://twitter.com/verstegenrick

View solution in original post

RickVerstegen · ‎09-26-2018

ERROR bindings.sso_binding Cannot log-in to SSO admin service, because of Error: (vmodl.fault.InvalidRequest) {

dynamicType = <unset>,

Looks like a bad SSO password regarding the logs.

Change the SSO password to a supported password.and restart the upgrade process.

_______________________________________________________________________________________________________

"Did you find this helpful? Let us know by completing this survey (takes 1 minute!)"

Was I helpful? Give a kudo for appreciation!
Blog: https://rickverstegen84.wordpress.com/
Twitter: https://twitter.com/verstegenrick

andreir · ‎09-27-2018

Thanks, Rick!

Not sure what was wrong with my old password, but I changed administrator@vsphere.local password to a random 10 char string and was able to proceed with the upgrade.

GBartsch · ‎05-22-2019

Do either of you have any idea as to what "Failed normalizing ip: psc04.vlab.local" in the OPs CLI logs actually means?

All

external PSC 6.0 upgrade to 6.5 fails - Internal error occurs during pre-upgrade checks