VIO plugin doesnt work, Openstack not showin

[MHyvonen]

I have the same problem which is addressed here https://communities.vmware.com/docs/DOC-28682?z=BYJ9FY

Though in this case, it doesn't show up in the /mob nor it doesn't tell anything specific about failing in the vcenter logs. My vcenter is an appliance (newest 5.5 version, updated today).

Openstack is mentioned in the vcenter only on these places:

find / -xdev -type f -print0 | xargs -0 grep -H "openstack"

Binary file /usr/lib/vmware-vpx/sps/resources/cachedb/sms-cache0.h2.db matches

Binary file /usr/lib/vmware-vpx/sps/resources/cachedb/sms-cache1.h2.db matches

/etc/services:openstack-id       35357/tcp    # OpenStack ID Service  [Rackspace_Hosting] [Ziad_Sawalha]

ESXi host is also running newest 5.5. VIO management server appliance boots up ok (though it sets up localhost.localdom system name, and there's fqdn + reverse configured during the installation)

Management server reboot doesn't solve the issue, neither does reloggin from web client. Both vcenter and management server are in the same private subnet ( /24) and can ssh to each other.

[yjia]

First please check if this doc can help you out of the trouble or not.

vCenter certificate issues when deploying VIO

It will be great if you can upload the management server log , so that we can check what happens on your server.

Here is how to get the logs.

1> login to the management server.

2> fetch the follow logs

/var/log/bootsequence.log

/var/log/oms/*.log

At your service

Yixing.

[MHyvonen]

Hi.

Here's the bootsequence.log: https://dl.dropboxusercontent.com/u/8766670/bootsequence.log.txt

oms/ directory is empty.

[yjia]

Ok, the bootsequence is all right.

Now Please try the follow steps and let me know if you still can't see any logs under the /var/log/oms directory.

a> In the VC client, please restart the vApp: VMware Integrated Openstack. ( first select ShutDown, after it completely shutdown, then select Power On)

b> wait sometime, say 5minutes, then check the oms directory.

Regards.

[MHyvonen]

No luck with oms logs after restart

root@localhost:/var/log/oms# uptime

09:08:05 up 7 min,  1 user,  load average: 0.00, 0.03, 0.04

root@localhost:/var/log/oms# ls -la

total 8

drwxr-x---  2 omsapp adm    4096 Mar  6 17:49 .

drwxrwxr-x 13 root   syslog 4096 May 13 09:00 ..

root@localhost:/var/log/oms#

[yjia]

Thanks for the retry.

Now I need your help to provide the follow files.

a> Please login you management server

b> Change to /var/log/upstart  directory

c> please zip all the oms* osvmw* and vpostgre* files

These files will help us to identify what happens on your server.

Thanks.

[MHyvonen]

There

https://dl.dropboxusercontent.com/u/8766670/logs.tar

Thanks for the very quick answers!

[yjia]

Thanks for those logs, from the log we can see that some thing prevent tomcat from start up on your server.

I need more logs to identify what it is.

Please help provide the follow files

a> login to management server

b> zip all the catalina* under the /opt/vmware/vio/tc-oms/logs  directory.

Thanks.

[yjia]

Also , please let me know the version of the VMware Integrated Openstack you are using( it will be great if you can also provide the build number, which you can find it at /opt/vmware/vio/etc/vio_system.properties  the first two item, vio.build and vio.version)

[MHyvonen]

Fresh install, couple of days old. Didn't apply the new patch though yet.

vio.build = 2567172

vio.version = 1.0.0

https://dl.dropboxusercontent.com/u/8766670/cata.tar.gz

[yjia]

Thanks for your cooperation!

From the log we can identify that there is something wrong with the preinstall process of the management server.

We need your help to provide the follow files for further analysis.

all the files under /opt/vmware/vio/etc

Regards,

Yixing.

[MHyvonen]

There u go

https://dl.dropboxusercontent.com/u/8766670/etc.tar

[yjia]

Thanks for the logs,

Please try the follow solution to see if it works.

a> edit the /opt/vmware/vio/ets/keystore.properties

change the vio.keystorePass=fsdjdALRALAv4KHohqVo   to    vio.keystorePass=ZUdCSQEUx8mNhxQER8Ob

b> edit the /opt/vmware/vio/tc-oms/conf/catalina.properties   scroll to the end of the file

change

vio.keystorePass=fsdjdALRALAv4KHohqVo   to    vio.keystorePass=ZUdCSQEUx8mNhxQER8Ob

c>Edit the /opt/vmware/vio/tc-osvmw/conf/catalina.properties   scroll to the end of the file

change

vio.keystorePass=fsdjdALRALAv4KHohqVo   to    vio.keystorePass=ZUdCSQEUx8mNhxQER8Ob

d> save the above modification.

e> Restart the vApp.

Then Let us wait some minutes (5)   and check the /var/log/oms directory to see if we can see any log files there.

[MHyvonen]

Still empty

[yjia]

Ok, if we still can not see any logs, I think may be you need to delete the vApp and redeploy it .

From the logs you provide to us, we find that some key items are inconsistent in your system. We are still try to find out what cause that happen. But ideally an reinstall should solve the problem.

Sorry for the inconvenient brings to you.

[MHyvonen]

Sorry for the troubles I brought to u

I deleted the vApp, redeployed it, patched it with 1.01 patch, restarted web client service on vcenter and *drumfill* there's now logs in the oms/ directory!

There's no OpenStack icon in the vcenter home view though (I relogged etc). So there's progress but we're not in goal yet.

Sadly I must "close the shop" for the day so I can't examine this further. I'll post oms logs here and I wait for your answer how things are looking, but I won't be tinkering more until tomorrow.

https://dl.dropboxusercontent.com/u/8766670/oms.tar

Cheers!

[yjia]

From the latest log, we can identify that the Management server is startup successfully.

Please try to clear your browser cache and relogin to see if you can see the icon.

If still not see the Icon, then please send me the follow files.

1> login  to your VC server.

2> change to the /var/log/vmware/vsphere-client/logs/ directory

3> collect all the vsphere_client_virgo* files.

Regards

[yjia]

Hi,

Can you please confirm whether the issue has been solved or not?

Regards,

Yixing.

[MHyvonen]

Good news. I destroyed the vapp, redeployed it and upgraded it to 1.01. And now it works!

Every step I made in the 2nd installation was exactly the same it was in the 1st round, I just didn't apply the update before.

This case is now solved.

[s_mcgarry]

HI All,

Has anyone else been experiencing this issue pls ?

I have tried re-installing the VIO OVF several times & then applied the patch, but each time has ended in the same result:

>OMS server successfully deployed and started

>No VIO plugin visible in the Web client for the VC, despite restarting the OMS VM + VC.

>If you log on to the machine, you can see the CPU consumption is constantly 40-50%

>If you look at /var/log/syslog, you see the following message continually looping:

May 26 06:37:21 vio-manager kernel: [310730.259140] init: oms main process (7506) terminated with status 1

May 26 06:37:21 vio-manager kernel: [310730.259153] init: oms main process ended, respawning

May 26 06:37:25 vio-manager kernel: [310733.458880] init: osvmw main process (7638) terminated with status 1

May 26 06:37:25 vio-manager kernel: [310733.458894] init: osvmw main process ended, respawning

May 26 06:37:26 vio-manager kernel: [310734.348785] init: oms main process (7653) terminated with status 1

May 26 06:37:26 vio-manager kernel: [310734.348798] init: oms main process ended, respawning

> There are no logs in /var/log/oms

> Content of the bootsequence log:

Fri May 22 16:22:50 UTC 2015 : Running firstboot scripts

Running commands from firstboot script /opt/vmware/vio/sysctl/firstboot.d//00-common

Creating SSH2 RSA key; this may take some time ...

Creating SSH2 DSA key; this may take some time ...

Creating SSH2 ECDSA key; this may take some time ...

Creating SSH2 ED25519 key; this may take some time ...

ssh stop/waiting

ssh start/running, process 1444

Running commands from firstboot script /opt/vmware/vio/sysctl/firstboot.d//02-setup-user

passwd: password expiry information changed.

groupadd: group 'viogrp' already exists

Group viogrp exists!

unable to write 'random state'

Generating public/private rsa key pair.

Your identification has been saved in /home/viouser/.ssh/id_rsa.

Your public key has been saved in /home/viouser/.ssh/id_rsa.pub.

The key fingerprint is:

a4:c1:c0:19:7c:fc:e9:8a:21:02:de:85:fe:6c:cd:1d root@vio-manager.viotest.lab.168.192.in-addr.arpa

The key's randomart image is:

+--[ RSA 2048]----+

|   oo+           |

|    +oo          |

|    ..o...       |

|.  . . +o        |

|o o . ..S        |

|...o.   .E       |

| . .ooo.. .      |

|    .+.o .       |

|    .            |

+-----------------+

Running commands from firstboot script /opt/vmware/vio/sysctl/firstboot.d//20-generate-keys

1. Generating keystore for OMS server...

keytool error: java.lang.RuntimeException: java.io.IOException: DNSName components must begin with a letter

   Exporting self-signed certificate...

keytool error: java.lang.Exception: Keystore file does not exist: /opt/vmware/vio/etc/oms.jks

   Generating the csr...

keytool error: java.lang.Exception: Keystore file does not exist: /opt/vmware/vio/etc/oms.jks

   Exporting pkcs12 store...

keytool error: java.io.FileNotFoundException: /opt/vmware/vio/etc/oms.jks (No such file or directory)

   Exporting private key...

Error opening input file /opt/vmware/vio/etc/oms.p12

/opt/vmware/vio/etc/oms.p12: No such file or directory

2. Generating password guard keystore...

generated certificate and private key in /opt/vmware/vio/etc

Running commands from firstboot script /opt/vmware/vio/sysctl/firstboot.d//30-generate-vault-pass

Running commands from firstboot script /opt/vmware/vio/sysctl/firstboot.d//48-configure_rsyslog

rsyslog stop/waiting

rsyslog start/running, process 1595

Fri May 22 16:22:54 UTC 2015 : Running everyboot scripts

Running commands from everyboot script /opt/vmware/vio/sysctl/everyboot.d//00-common

Running commands from everyboot script /opt/vmware/vio/sysctl/everyboot.d//55-set-vami-guestinfo

Running commands from everyboot script /opt/vmware/vio/sysctl/everyboot.d//70-configure_ntp

* Stopping NTP server ntpd

   ...done.

22 May 16:23:10 ntpdate[1623]: adjust time server 192.168.30.5 offset 0.448450 sec

* Starting NTP server ntpd

   ...done.

Running commands from everyboot script /opt/vmware/vio/sysctl/everyboot.d//71-update-exec-context

Setting up execution env: export VC information to vc.properties

Running commands from everyboot script /opt/vmware/vio/sysctl/everyboot.d//73-configure_vcsso

Intializing registration provider...

Getting SSL certificates for https://192.168.32.10:7444/lookupservice/sdk

Anonymous execution

Successfully saved SSO locations and certificates

Return code is: Success

Prepare the authentication component

SSO Authentication is enabled now. Please restart the OMS web server to complete the procedure by the command 'stop/start oms'.

Running commands from everyboot script /opt/vmware/vio/sysctl/everyboot.d//75-configure-tc-oms

Running commands from everyboot script /opt/vmware/vio/sysctl/everyboot.d//76-configure-tc-osvmw

Running commands from everyboot script /opt/vmware/vio/sysctl/everyboot.d//100-boot-complete

Sending boot-complete signal to upstart

Any help would be greatly appreciated as this is now stopping an important project from progressing ...

Cheers

Steve