6.0 > 6.7 fails at starting Identity Service

[yannbizeul]

Relevant exception is as follows. You will notice that for some reason the URL has a newline character after the host name, visible on the first line in the arguments, and on line 12 as well.

How is it passed to vmidentity-firstboot.py and how can I fix it?

2018-10-07T16:10:02.506Z  Running command: ['/usr/java/jre-vmware/bin/java', '-cp', '/opt/vmware/lib64/*:/usr/lib/vmware-sso/commonlib/*:.:*', '-Dlog4j.configuration=initls-log4j.properties', '-Dvmware.log.dir=/var/log/vmware/sso/', '-XX:ErrorFile=/var/log/vmware/sso/hs_err_stsinstaller_pid%p.log-XX:HeapDumpPath=/var/log/vmware/sso/', 'com.vmware.vim.lookup.tools.InitializeLookupService', '--cert-path', '/etc/vmware-sso/keys/machine.crt', '--host-name', '10.65.180.7\n', '--http-port', '443']
2018-10-07T16:10:04.847Z  Done running command
2018-10-07T16:10:04.848Z  >>>>stderr:
2018-10-07T16:10:04.848Z  Picked up JAVA_TOOL_OPTIONS: -Xms32M -Xmx128M
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/vmware/lib64/slf4j-log4j12-1.7.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/vmware/lib64/log4j-slf4j-impl-2.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Failed to initialize lookup service: java.lang.IllegalArgumentException: Illegal character in authority at index 8: https://10.65.180.7
/sts/STSService/vsphere.local
java.lang.IllegalArgumentException: Illegal character in authority at index 8: https://10.65.180.7
/sts/STSService/vsphere.local
  at java.net.URI.create(URI.java:852)
  at com.vmware.vim.lookup.tools.InitializeLookupService.createURI(InitializeLookupService.java:373)
  at com.vmware.vim.lookup.tools.InitializeLookupService.createServiceEndpoint(InitializeLookupService.java:300)
  at com.vmware.vim.lookup.tools.InitializeLookupService.createServiceEndpointList(InitializeLookupService.java:311)
  at com.vmware.vim.lookup.tools.InitializeLookupService.createNewLookupService(InitializeLookupService.java:342)
  at com.vmware.vim.lookup.tools.InitializeLookupService.main(InitializeLookupService.java:150)
Caused by: java.net.URISyntaxException: Illegal character in authority at index 8: https://10.65.180.7
/sts/STSService/vsphere.local
  at java.net.URI$Parser.fail(URI.java:2848)
  at java.net.URI$Parser.parseAuthority(URI.java:3186)
  at java.net.URI$Parser.parseHierarchical(URI.java:3097)
  at java.net.URI$Parser.parse(URI.java:3053)
  at java.net.URI.<init>(URI.java:588)
  at java.net.URI.create(URI.java:850)
  ... 5 more

[SupreetK]

Looks like there is an additional character in the SSO name. Can you share the output of the commands <cat /etc/vmware-sso/hostname.txt> and <file /etc/vmware-sso/hostname.txt>?

Cheers,

Supreet

[yannbizeul]

Thanks for helping.

I'm doing this on the source VM as I scratched the destination VM to retry.

It doesn't look like there is additional characters here.

prod_vcenter6:~ # cat /etc/vmware-sso/hostname.txt
10.65.180.7
prod_vcenter6:~ # file /etc/vmware-sso/hostname.txt
/etc/vmware-sso/hostname.txt: ASCII text

[yannbizeul]

On the destination VM these files don't exists

[SupreetK · Accepted solution]

Can you execute the attached python script on the source vCSA and share the output? If you place the script under /tmp, the command would be <python /tmp/Test_Script.py>. Hope you have already taken a snapshot of the vCenter appliance. If not, this is the time.

Cheers,

Supreet

[yannbizeul]

Now, that's interesting :

^10.65.180.7
^

And of course hexdump confirms this :

vcenter:~ # cat /etc/vmware-sso/hostname.txt |hexdump -C
00000000  31 30 2e 36 35 2e 31 38  30 2e 37 0a              |10.65.180.7.|
0000000c

Damn it... I though I made sure already but I guess I wasn't on the right VM or the wrong file.

Let me fix the file and try again.

[SupreetK]

Cool! Attached is a script that will help you fix the file. Once it is fixed, running the file command should report something like '... ASCII text, with no line terminators'. Once this shows up, retry the upgrade and let me know.

Cheers,

Supreet

[SupreetK]

yannbizeul​ Any update on this? Did the upgrade work after fixing the file?

Cheers,

Supreet

[yannbizeul]

Hi Gurpreet,

Yes, I was able to finish the upgrade this morning.

I got really lucky because during my investigations I decided to upgrade to the latest 6.0, and It happens you cannot upgrade latest 6.0 to 6.5 or 6.7, something related to a PostgreSQL security fix.

So I rolled back to the original 6.0 version I had in a snapshot, and then ran the upgrade after fixing the file.

Everything is good now! Thanks