Upgrade form 8.10.2 to 8.12 failed with "name 'ec' is not defined"

[IRIX201110141]

root@syslog [ ~ ]# /usr/lib/loginsight/application/sbin/loginsight-pak-upgrade.py /tmp/VMware-vRealize-Log-Insight-8.12.0-21618456.pak
Certificate verified:  VMware-vRealize-Log-Insight.cert: C = US, ST = California, L = Palo Alto, O = "VMware, Inc."
error 18 at 0 depth lookup:self signed certificate
OK
Signature of the manifest validated: Verified OK

Current version is 8.10.2-21145187 and upgrade version is 8.12.0-21618456. Version Check successful!
Available Disk Space at /tmp: 7623974912
Disk Space Check successful!
Available Disk Space at /storage/core: 17117868032
Disk Space Check successful!
Available Disk Space at /storage/var: 10179022848
Disk Space Check successful!
Checksum Check successful!
Attempting to upgrade to version 8.12.0-21618456
Error while running upgrade: name 'ec' is not defined

On other instances the upgrade was sucessfull but 2/3 of my Content Packs went away an half of them cant reinstalled like MSSQl, Veeam and others. I get a success msg but its not visible in the left menu or later marked as installed.

2023-04-22 20:33:01,557 upgrade-driver INFO INFO: Running /storage/core/upgrade/kexec-li - Resize|Partition|Boot
... Starting to run kexec-li script ...
Failed moving upgrade token file
Reading and saving /etc/ssh/sshd_config
Reading and saving old ssh keys if key based Authentication is enabled
cp: cannot stat '/root/.ssh//id_rsa': No such file or directory
cp: cannot stat '/root/.ssh//id_rsa.pub': No such file or directory
cp: cannot stat '/root/.ssh//known_hosts': No such file or directory
Reading and saving /etc/hosts
Reading and saving ssh host keys
Reading and saving /var/lib/loginsight-agent/liagent.ini
Reading and saving hostname
Reading and saving old cassandra keystore
Failed copying /usr/lib/loginsight/application/lib/apache-cassandra-*/conf/keystore*
Reading and saving old default keystore
Reading and saving old default truststore
Reading and saving old tomcat configs
chmod ing /storage/core/upgrade/vmdk-extracted-root/usr/lib/loginsight/application/etc/3rd_config/keystore*
chmod ing /storage/core/upgrade/vmdk-extracted-root/usr/lib/loginsight/application/etc/truststore*
Reading and saving old loginsight.conf
Reading and saving old password in /etc/shadow
Root password info root P 04/22/2023 0 3650 3650 -1
Root password change date is 04/22/2023
Root password is set. Password reset will not be required on first login.
Reading and saving /etc/fstab
Reading and saving cacerts
Copying java.security to java.security.old
Reading and saving network configs
Reading and saving resolv.conf
Checking for certificate renewal
Current certificate fingerprint: C0:86:0E:33:24:2C:AD:BB:01:C6:94:C6:BF:B8:40:5A:74:CF:86:AD
Cassandra and tomcat certificate fingerprints are different. Updating...
Custom certificate lacks 'SSL client' purpose
2023-04-22 20:33:31,094 upgrade-driver INFO Exception occured!!!
2023-04-22 20:33:31,094 upgrade-driver INFO Uninstalling upgrade-image.rpm
2023-04-22 20:33:31,219 upgrade-driver INFO Restarting Log Insight
2023-04-22 20:33:47,374 upgrade-driver INFO Finished running upgrade-script!!!!
2023-04-22 20:33:47,387 loginsight-pak-upgrade INFO Starting 'upgrade-driver' script ...
Start processing the  manifest file ...
Log Insight TO_VERSION is manifest file is 8.12.0-21618456
Parsed version is 8.12.0-21618456
Creating file /storage/core/upgrade-version to store upgrade version.
The file /storage/core/upgrade-version is created successfully.
Start upgrading cassandra sstable schema...
Cassandra sstable schema upgrading done.
Start processing key list ...
Start processing rpm list ...
Rpm by name upgrade-image-8.12.0-21618456.rpm
Exception encountered during key generation, continuing: Unicode-objects must be encoded before hashing
INFO: Running /storage/core/upgrade/kexec-li - Resize|Partition|Boot
ERROR: Failed to run /storage/core/upgrade/kexec-li
Exception occured!!!
Uninstalling upgrade-image.rpm
Restarting Log Insight
Finished running upgrade-script!!!!
2023-04-22 20:33:47,387 loginsight-pak-upgrade INFO name 'ec' is not defined
2023-04-22 20:33:47,387 loginsight-pak-upgrade ERROR Error while running upgrad

Regards,
Joerg

[Braack]

Hello Jörg, I'm hitting the same error while trying to go from 8.10 to 8.12 in a three node cluster.

Did you find a solution?

Thanks,

Peter

[Charvo]

Reset Certificate to default certificate and try upgrade from UI page.

[IRIX201110141 · Accepted solution]

Hi Peter,

yes i got it working. I change back to the VMware Default SSL cert and than the upgrade works flawlessly.

I'm more concern about all the Contentpacks[*] which disapers after the upgrade. Even re-installing them from the store or importing doesnt work. Iam unable to find something in the logs. My Workaround was to increase the version number and changing the namespace and importing them again.

* Even the VMware MSSQL is effected

Regards,
Joerg

[Braack]

Thanks! I changed back to default SSL, but same error.

Will open a SR with VMware.

Parsed version is 8.12.0-21618456
Creating file /storage/core/upgrade-version to store upgrade version.
The file /storage/core/upgrade-version is created successfully.
Start upgrading cassandra sstable schema...
Cassandra sstable schema upgrading done.
Start processing key list ...
Start processing rpm list ...
Rpm by name upgrade-image-8.12.0-21618456.rpm
Exception encountered during key generation, continuing: Unicode-objects must be encoded before hashing
INFO: Running /storage/core/upgrade/kexec-li - Resize|Partition|Boot
ERROR: Failed to run /storage/core/upgrade/kexec-li
Exception occured!!!
Uninstalling upgrade-image.rpm
Restarting Log Insight
Finished running upgrade-script!!!!
2023-04-25 08:20:22,667 loginsight-pak-upgrade INFO name 'ec' is not defined
2023-04-25 08:20:22,667 loginsight-pak-upgrade ERROR Error while running upgrade
Traceback (most recent call last):
File "/usr/lib/loginsight/application/sbin/loginsight-pak-upgrade.py", line 516, in main
raise UpgradeError(err)
UpgradeError: "name 'ec' is not defined"
2023-04-25 08:20:22,669 loginsight-pak-upgrade INFO Done!

[IRIX201110141]

Did you try to reboot after the SSL change?

[SanderHogewerf]

Can i also do this from CLI. My UI crashed, and can't get it back. 

[Braack]

Yes, you can use the CLI:

i.e. /usr/lib/loginsight/application/sbin/loginsight-pak-upgrade.py /tmp/VMware-vRealize-Log-Insight-8.12.0-21618456.pak

See https://kb.vmware.com/s/article/83225.

[SanderHogewerf]

I meant changing the Cert, because i got exactly the same problem as you.

[Braack]

Not sure yet. Will keep you informed as soon as I get something back from VMware.

[mkfmevntm]

thanks to the comments here I got the update working after resetting the SSL certificate (without reboot).

After the update I had to re-create the custom cert with a new csr including "extendedKeyUsage = serverAuth, clientAuth".
Importing the new cert seemed stuck at first, I waited some more minutes and did a reboot (Snapshot first!), after the reboot the new cert was installed.