VMware Cloud Community
Cacciari
Contributor
Contributor

vCloud Director installation freezes.

Hi,

Something very strange is happening.

Installation of vcd freezes  when loading .sql files in. Like this:

Enter the database service name [default=orcl]: test
Enter the database username: test
Enter the database password: *****
Connecting to the database: jdbc:oracle:thin:@192.168.1.30:1521/test

loading /opt/vmware/cloud-director/db/oracle/NewInstall.sql
[405 statements]
...............................................................................................[100]
...............................................................................................[200]
...............................................................................................[300]
...............................................................................................[400]
.....[405]

loading /opt/vmware/cloud-director/db/oracle/NewInstall_Funcs.sql
[162 statements]
......................................................................

and then it stops with these errors:

2011-01-31 12:00:28,571 | DEBUG    | main                      | ConfigAgent                    | VCLOUD_HOME is /opt/vmware/cloud-director |
2011-01-31 12:00:28,572 | DEBUG    | main                      | ConfigAgent                    | SQL files will be read from /opt/vmware/cloud-director/db |
2011-01-31 12:00:28,573 | DEBUG    | main                      | ConfigAgent                    | Using default properties file. |
2011-01-31 12:00:28,573 | DEBUG    | main                      | ConfigAgent                    | Properties file is /opt/vmware/cloud-director/etc/global.properties |
2011-01-31 12:00:28,573 | INFO     | main                      | ConfigAgent                    | Reading responses from file: /opt/vmware/cloud-director/etc/responses.properties |
2011-01-31 12:00:28,573 | INFO     | main                      | ConfigAgent                    | Reading configuration from file: /opt/vmware/cloud-director/etc/global.properties |
2011-01-31 12:00:28,574 | INFO     | main                      | ConfigAgent                    | System Version=2 |
2011-01-31 12:00:29,674 | DEBUG    | main                      | EncryptionManager              | EncryptionManager configured with system key. |
2011-01-31 12:00:29,675 | DEBUG    | main                      | EncryptionManager              | EncryptionManager has been loaded successfully. |
2011-01-31 12:02:22,285 | INFO     | main                      | Db                             | Proceeding to initialize database. |
2011-01-31 12:18:47,014 | ERROR    | main                      | ConfigAgent                    | Error while communicating with the database: Closed Connection |
java.sql.SQLRecoverableException: Closed Connection
        at oracle.jdbc.driver.PhysicalConnection.commit(PhysicalConnection.java:3640)
        at oracle.jdbc.driver.PhysicalConnection.commit(PhysicalConnection.java:3680)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at com.vmware.vcloud.common.jdbcpool.ProxyConnection.invoke(ProxyConnection.java:129)
        at $Proxy0.commit(Unknown Source)
        at com.vmware.vcloud.configure.Db.createTables(Db.java:288)
        at com.vmware.vcloud.configure.Db.maybeInitialize(Db.java:531)
        at com.vmware.vcloud.configure.ConfigAgent.configureDatabase(ConfigAgent.java:1666)
        at com.vmware.vcloud.configure.ConfigAgent.start(ConfigAgent.java:425)
        at com.vmware.vcloud.configure.ConfigAgent.main(ConfigAgent.java:320)

Ok, it says closed connection but why? Do you guys have an idea of what should I check in order to make that work? Until the freezing everything runs extremely fast. I just don't know what to do from now.

Any thought is useful.

Reply
0 Kudos
6 Replies
sjoglekar
Enthusiast
Enthusiast

Is there a firewall between the cell and the DB? What version of the DB are you using?

Thanks,

Sangeeta

Reply
0 Kudos
Cacciari
Contributor
Contributor

sjoglekar wrote:

Is there a firewall between the cell and the DB?

No.

What version of the DB are you using?

Oracle 11g.

Please note that I actually got connection from this Linux machine to the Oracle database. You can see that the first .sql file is completely executed. Should that be a firewall problem nothing would be executed at all because no connection would be possible.

If I run scripts manually using sqlplus at the other side (the oracle database), database does not complains, but I cannot get the service up and running at vcloud server side because the ...../configure script does not runs entirely.

Even if I comment parts on script to avoid running the .sql files (because they have been executed already manually by me, as just mentioned), service does not works properly.

Reply
0 Kudos
sjoglekar
Enthusiast
Enthusiast

I am not sure, but you might be hitting this oracle issue -

http://www.usn-it.de/index.php/2009/02/20/oracle-11g-jdbc-driver-hangs-blocked-by-devrandom-entropy-...

This link suggests a solution in the last post:

http://kr.forums.oracle.com/forums/thread.jspa?threadID=941911

Try the suggested:

rm /dev/random
ln -s /dev/urandom /dev/random

...which worked but it is gone after every reboot.

If this approach works, then we are sure that the connection close is because of this issue. We can add "-Djava.security.egd=file:///dev/urandom" in the JVM arguments once we are sure about this issue.

Thanks,

Sangeeta

Reply
0 Kudos
Cacciari
Contributor
Contributor

Try the suggested:

rm /dev/random
ln -s /dev/urandom /dev/random

...which worked but it is gone after every reboot.

Thanks,

Sangeeta

Tried this but no success. Also I guess that a random element would not cause the installation to stop at the very same spot every time. It always lock on the same stage. After 70 statements of the second file:

loading /opt/vmware/cloud-director/db/oracle/NewInstall_Funcs.sql
[162 statements]
...................................................................... <<<- 70 dots here.

I´ve rerun a fresh new installation of Linux. Centos 5.5 64-bit.

1) Rechecked DNS PTR and A. works

VShield, vcloud, esx, oracle, vcenter. Everyone can resolve everyone.

2) Rechecked certificates with

[root@localhost bin]# ./keytool -storetype JCEKS -storepass blablabla -keystore certificates.ks -list

Keystore type: JCEKS
Keystore provider: SunJCE

Your keystore contains 2 entries

consoleproxy, Feb 1, 2011, PrivateKeyEntry,
Certificate fingerprint (MD5): BL:AB:LA:BL:AB:LA:BL:AB:LA:BL:AB:LA:BL:AB:LA
http, Feb 1, 2011, PrivateKeyEntry,
Certificate fingerprint (MD5): BL:AB:LA:BL:AB:LA:BL:AB:LA:BL:AB:LA:BL:AB:LA

3) Cleared database, according to this VMware KB:

kb.vmware.com/kb/1025695

Resetting the vCloud Director Database After a Failed Installation

4) On script re-run, tried to link to database with name and IP. Both fails.

Additional notes:

5) This vcloud is in one datacenter - the database is in another one.

PING 192.168.23.8 (192.168.23.8) 56(84) bytes of data.
64 bytes from 192.168.23.8: icmp_seq=1 ttl=62 time=32.5 ms

6) We have a vCenter/ESX datacenter split on the same situation and it works perfectly.

7) Everything is under the "2 seconds" time difference as the installation manual states.

I must be missing something. There must be a very obvious workaround for this. I read instruction manual several times and everything is there. but it just doesn't works. I don't know why.

Reply
0 Kudos
sjoglekar
Enthusiast
Enthusiast

Can you see any errors in the /opt/vmware/logs/vmware-vcd-debug.log ?

Can you attach all logs from this folder?

Sangeeta

Reply
0 Kudos
Cacciari
Contributor
Contributor

As time matters more than boiling by brain debugging, i've ditched the 64bit oracle and replaced by a 32bit version.

It worked flawlessly.

That will remain a mistery for me - perhaps something related to Oracle 11g R2, but I won't investigate that anymore.

Thanks for your help, Sangeeta

Reply
0 Kudos