Re: vCAC 6.0.x to vCAC 6.1 Upgrade Fails

UberGeek1 · ‎09-09-2014

So having this issue when trying to upgrade the database on the IaaS server. Following through the Upgrade Guide, I'm getting the following errors in the upgrade log:

Project "C:\temp\DBUpgrade\Liquibase.xml" on node 1 (default targets).

RunLiquibase:

"c:\Program Files\Java\jre7\bin\java.exe" -Djava.library.path="C:\temp\DBUpgrade" -cp "C:\temp\DBUpgrade\liquibase.jar;C:\temp\DBUpgrade\jtds-1.3.1.jar" liquibase.integration.commandline.Main --driver=net.sourceforge.jtds.jdbc.Driver --url=jdbc:jtds:sqlserver://localhost/"vCACDBName";instance=vCACDBName releaseLocks

Liquibase releaseLocks Failed: liquibase.exception.DatabaseException: java.sql.SQLException: Network error IOException: Connection refused: connect

C:\temp\DBUpgrade\Liquibase.xml(28,5): error MSB3073: The command ""c:\Program Files\Java\jre7\bin\java.exe" -Djava.library.path="C:\temp\DBUpgrade" -cp "C:\temp\DBUpgrade\liquibase.jar;C:\temp\DBUpgrade\jtds-1.3.1.jar" liquibase.integration.commandline.Main --driver=net.sourceforge.jtds.jdbc.Driver --url=jdbc:jtds:sqlserver://localhost/"vCACDBName";instance=vCACDBName releaseLocks" exited with code -1.

Done Building Project "C:\temp\DBUpgrade\Liquibase.xml" (default targets) -- FAILED.

Build FAILED.

"C:\temp\DBUpgrade\Liquibase.xml" (default target) (1) ->

(RunLiquibase target) ->

C:\temp\DBUpgrade\Liquibase.xml(28,5): error MSB3073: The command ""c:\Program Files\Java\jre7\bin\java.exe" -Djava.library.path="C:\temp\DBUpgrade" -cp "C:\temp\DBUpgrade\liquibase.jar;C:\temp\DBUpgrade\jtds-1.3.1.jar" liquibase.integration.commandline.Main --driver=net.sourceforge.jtds.jdbc.Driver --url=jdbc:jtds:sqlserver://localhost/"vCACDBName";instance=vCACDBName releaseLocks" exited with code -1.

Sincerely, Jody L. Whitlock

UberGeek1 · ‎09-09-2014

Figured it out; in SQL Server Configuration Manager, under SQL Server Network Configuration, it looks like TCP is enabled, but don't be fooled. Open the Properties for TCP and under IP1, make sure "Enabled" is selected and 1433 is in the TCP Port field.

Sincerely, Jody L. Whitlock

JohnVMware123 · ‎09-09-2014

I've got an error at vCAC appliance upgrade:

Failed to install updates(Error while running package installation) on Tuesday, September 9, 2014 12:11:35 PM GMT-07:00

Any idea?

JohnVMware123 · ‎09-09-2014

Hi,

Here is the catalina.2014-09-09.log

09-Sep-2014 19:31:25.089 SEVERE [localhost-startStop-8] org.apache.catalina.loader.WebappClassLoader.checkThreadLocalMapForLeaks The web application [/shell-ui-app] created a ThreadLocal with key of type [com.google.inject.internal.InjectorImpl$1] (value [com.google.inject.internal.InjectorImpl$1@7fda4b89]) and a value of type [java.lang.Object[]] (value [[Ljava.lang.Object;@340967b]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

09-Sep-2014 19:31:25.096 SEVERE [localhost-startStop-8] org.apache.catalina.loader.WebappClassLoader.checkThreadLocalMapForLeaks The web application [/shell-ui-app] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@55766e8]) and a value of type [com.google.javascript.jscomp.Tracer.ThreadTrace] (value []) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

09-Sep-2014 19:31:25.097 SEVERE [localhost-startStop-8] org.apache.catalina.loader.WebappClassLoader.checkThreadLocalMapForLeaks The web application [/shell-ui-app] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@55766e8]) and a value of type [com.google.javascript.jscomp.Tracer.ThreadTrace] (value []) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

09-Sep-2014 19:31:25.097 SEVERE [localhost-startStop-8] org.apache.catalina.loader.WebappClassLoader.checkThreadLocalMapForLeaks The web application [/shell-ui-app] created a ThreadLocal with key of type [com.google.inject.internal.InjectorImpl$1] (value [com.google.inject.internal.InjectorImpl$1@7fda4b89]) and a value of type [java.lang.Object[]] (value [[Ljava.lang.Object;@52fb4088]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

09-Sep-2014 19:31:25.158 SEVERE [localhost-startStop-6] org.apache.catalina.loader.WebappClassLoader.clearReferencesThreads The web application [/advanced-designer-service] appears to have started a thread named [Thread-41] but has failed to stop it. This is very likely to create a memory leak.

09-Sep-2014 19:31:25.208 INFO [Thread-4] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler ["ajp-bio-127.0.0.1-8009"]

09-Sep-2014 19:31:25.214 INFO [Thread-4] org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler ["ajp-bio-127.0.0.1-8009"]

Thanks in advance

stvkpln · ‎09-09-2014

Catalina won't help you for the upgrade.. it doesn't use tcServer. How are you doing the update? Did you download the update repo iso? If so, make sure the checksum for the file matches, or it may have gotten corrupted on download..

-Steve

JohnVMware123 · ‎09-09-2014

Yea, through ISO. Just checked checksum. they look okay. I've tried with 2 downloads, same error. services has been shutdown and identity appliance is up and running. did I miss anything here?

UberGeek1 · ‎09-09-2014

I thought I had it, but nope. I did get the dbupgrade to complete but then the IaaS components choke.

Sincerely, Jody L. Whitlock

UberGeek1 · ‎09-09-2014

I'm assuming you're on 6.0.1.1 and you are using g the repo iso? I had no issues with the appliances completing the upgrade, at least I think since I haven't logged back in yet.

Sincerely, Jody L. Whitlock

JohnVMware123 · ‎09-10-2014

I've used CDROM updates. Downloaded correctly but still getting same error

stvkpln · ‎09-10-2014

One thing to do is run the update while you're SSH'ed into the appliance and tail the following file: /opt/vmware/var/log/vami/updatecli.log

That should give you an idea of where the failure is occurring and give you a path forward (hopefully).

-Steve

JohnVMware123 · ‎09-10-2014

This is what I'm getting from updatecli.log:

014-09-10 18:37:29 /etc/bootstrap/postupdate.d/90-set-guestinfo done, status: 0

+ log 'main bootstrap postupdate done'

++ date '+%Y-%m-%d %H:%M:%S'

+ echo '2014-09-10 18:37:29 main bootstrap postupdate done'

2014-09-10 18:37:29 main bootstrap postupdate done

Restoring services state from /var/lib/vcac/services-cfg-chkconfig.state

apache2 on (was apache2 on)

ntp on (was ntp on)

sshd on (was sshd on)

vcac-server on (was vcac-server on)

vco-configurator off (was vco-configurator off)

vco-server off (was vco-server off)

vpostgres on (was vpostgres on)

Restoring ssh-cfg PermitRootLogin

PermitRootLogin yes (was PermitRootLogin yes)

Validating...

apache2 on

ntp on

sshd on

vcac-server on

vco-configurator off

vco-server off

vpostgres on

# ssh-cfg PermitRootLogin yes

vpostgres on

Starting VMware vPostgres

Last login: Wed Sep 10 18:37:06 EDT 2014

waiting for server to start.... done

server started

Last login: Wed Sep 10 18:37:30 EDT 2014

Failed with status of 4 while installing version 6.1.0.0

VM version is still 6.0.1.1

10/09/2014 22:37:32 [INFO] Update status: Done post-install scripts

10/09/2014 22:37:32 [INFO] Update status: Running VMware tools reconfiguration

10/09/2014 22:37:32 [INFO] Running /opt/vmware/share/vami/vami_reconfigure_tools

vmware-toolbox-cmd is /usr/bin/vmware-toolbox-cmd

Configuring VAMI VMware tools service wrapper.

vmware-toolbox-cmd is /usr/bin/vmware-toolbox-cmd

10/09/2014 22:37:32 [INFO] Update status: Done VMware tools reconfiguration

10/09/2014 22:37:32 [INFO] Update status: Error while running package installation

10/09/2014 22:37:32 [ERROR] Failure: updatecli exiting abnormally

10/09/2014 22:37:32 [INFO] Install Finished

Version - 6.0.1.1 Build 1768531

Description - vCAC Appliance 6.0.1.1

See the vCloud Automation Center Documentation web page at

http://www.vmware.com/info?id=1277

for installation instructions and release notes for vCAC 6.0.1.1.

JohnVMware123 · ‎09-10-2014

In logs: It saying can't create etc/bootstrap/preupdate.d/* not found', Did I miss anything before I do start upgrade?

og 'main bootstrap preupdate started'

++ date '+%Y-%m-%d %H:%M:%S'

+ echo '2014-09-10 19:47:08 main bootstrap preupdate started'

2014-09-10 19:47:08 main bootstrap preupdate started

+ for script in '"${BOOTSTRAP_DIR}"/*'

+ echo

+ '[' '!' -e '/etc/bootstrap/preupdate.d/*' ']'

+ log '/etc/bootstrap/preupdate.d/* not found'

++ date '+%Y-%m-%d %H:%M:%S'

+ echo '2014-09-10 19:47:08 /etc/bootstrap/preupdate.d/* not found'

2014-09-10 19:47:08 /etc/bootstrap/preupdate.d/* not found

+ log 'main bootstrap preupdate done'

++ date '+%Y-%m-%d %H:%M:%S'

+ echo '2014-09-10 19:47:08 main bootstrap preupdate done'

2014-09-10 19:47:08 main bootstrap preupdate done

+ '[' 6.0.1.1 '<' 6.1 ']'

+ sh /usr/lib/vcac/tools/preupdate/check_no_messages.sh

2014-09-10 19:47:08 host: '127.0.0.1' port: '5432' db: 'vcac' user: 'vcac'

2014-09-10 19:47:08 undeploy shell-ui-app

2014-09-10 19:47:23 Messages count=0, Status=0, msginfo=0||||

2014-09-10 19:47:23 there are 0 messages. Waiting 30 sec.

2014-09-10 19:47:33 Messages count=0, Status=0, msginfo=0||||

2014-09-10 19:47:33 there are 0 messages. Waiting 20 sec.

2014-09-10 19:47:43 Messages count=0, Status=0, msginfo=0||||

2014-09-10 19:47:43 there are 0 messages. Waiting 10 sec.

2014-09-10 19:47:53 Messages count=0, Status=0, msginfo=0||||

2014-09-10 19:47:53 there are 0 messages. Waiting 0 sec.

2014-09-10 19:48:03 Messages count=0, Status=0, msginfo=0||||

2014-09-10 19:48:03 no message queue changes in the timeout period

2014-09-10 19:48:03 stopping server

Stopping tcServer

ERROR Cannot find /var/lib/vcac/tcruntime-ctl.sh or it's not executable.

ERROR One of these files is needed to run this program

2014-09-10 19:48:03 tcServer stopped

2014-09-10 19:48:03 Messages count=0, Status=0, msginfo=0||||

2014-09-10 19:48:03 no messages after server stopped

+ sh /usr/lib/vcac/tools/preupdate/services-cfg.sh save

Saving services state to /var/lib/vcac/services-cfg-chkconfig.state

Saving ssh PermitRootLogin

apache2 on

ntp on

stvkpln · ‎09-10-2014

Maybe try rebooting (if you haven't done that), and trying again. Beyond that, my only recommendation would be to open an SR with support and work it that way. Make sure and take a log bundle of the appliance and send it as part of the upload so they can review and (hopefully) provide insight.

-Steve

PMors · ‎12-12-2014

I had similar output in my logfile and tried a ton off differnet things...

Only difference, I was going from 6.1 to 6.2.

At the end the solution for me has been chosing a differnet java version and at the same time a different OS that would run the dbupgrade.

Same installation procedure with the two Java versions

Java.exe Fileversion 8.025.18 on Server 2012 ---> not working

Java.exe Fileversion 7.0.710.14 on WIn 7 ---> work like charme

Regards

Peter

mikeslee · ‎12-13-2014

Agreed. That's what did it for me too. Backleveling to Java 7.Verify JAVA_HOME Env. Path is also set to: C:\Program Files\Java\jre7

RaymundoEC · ‎06-23-2015

Just to add a comment, it works only using JRE 1.7 so you have to un-install JRE 1.8, in my case to repoint to JRE 1.7 didn't work.

regards

+vRay

All

vCAC 6.0.x to vCAC 6.1 Upgrade Fails