VMware Cloud Community
FSvcoe
Enthusiast
Enthusiast

Error with populating config element-vCenter

All,

We deployed the self service portal some time ago, and it has run well up until recently. When attempting to run the Populate Configuration Element workflow, it fails and we see the following in the server.log.

-------------------------------------------------------------------

2014-02-10 16:12:30.679-0500 ERROR [VimClusterComputeResource] getProperty() [domain\username@https://myvcenter01.company.com:443/sdk/Main#664ce898, ClusterComputeResource<domain-c132812>] --> Property 'name' does not exist

com.vmware.vmo.plugin.vi4.model.NoSuchPropertyException

at com.vmware.vmo.plugin.vi4.model.VimContentManagedObject.addNewPropertyFilter(VimContentManagedObject.java:247)

  at com.vmware.vmo.plugin.vi4.model.VimContentManagedObject.getPropertyContent(VimContentManagedObject.java:298)

  at com.vmware.vmo.plugin.vi4.model.VimContentManagedObject.getProperty(VimContentManagedObject.java:376)

  at com.vmware.vmo.plugin.vi4.model.VimManagedEntity.getName(VimManagedEntity.java:124)

  at sun.reflect.GeneratedMethodAccessor221.invoke(Unknown Source)

  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

  at java.lang.reflect.Method.invoke(Method.java:597)

  at ch.dunes.vso.sdk.SDKFinder.convertToResult(SDKFinder.java:405)

  at ch.dunes.vso.sdk.SDKFinder.findForId(SDKFinder.java:112)

  at ch.dunes.vso.sdk.ModulesFactory.findForId(ModulesFactory.java:447)

  at ch.dunes.vso.sdk.mbean.ScriptingSDK.findForId(ScriptingSDK.java:103)

  at sun.reflect.GeneratedMethodAccessor218.invoke(Unknown Source)

  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

  at java.lang.reflect.Method.invoke(Method.java:597)

  at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)

  at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)

  at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)

  at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)

  at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)

  at ch.dunes.model.ejb.server.helper.SDKModulesHelper.invokeMBean(SDKModulesHelper.java:37)

  at ch.dunes.model.ejb.server.helper.SDKModulesHelper.findForId(SDKModulesHelper.java:180)

  at ch.dunes.model.ejb.server.sessions.VSOFactoryBean.findForId(VSOFactoryBean.java:1053)

  at sun.reflect.GeneratedMethodAccessor217.invoke(Unknown Source)

  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

  at java.lang.reflect.Method.invoke(Method.java:597)

  at org.jboss.invocation.Invocation.performCall(Invocation.java:359)

  at org.jboss.ejb.StatefulSessionContainer$ContainerInterceptor.invoke(StatefulSessionContainer.java:598)

  at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:168)

  at org.jboss.resource.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:158)

  at org.jboss.ejb.plugins.CallValidationInterceptor.invoke(CallValidationInterceptor.java:63)

  at org.jboss.ejb.plugins.AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:121)

  at org.jboss.ejb.plugins.AbstractTxInterceptorBMT.invokeNext(AbstractTxInterceptorBMT.java:173)

  at org.jboss.ejb.plugins.TxInterceptorBMT.invoke(TxInterceptorBMT.java:77)

  at org.jboss.ejb.plugins.StatefulSessionInstanceInterceptor.invoke(StatefulSessionInstanceInterceptor.java:333)

  at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:205)

  at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:138)

  at org.jboss.ejb.SessionContainer.internalInvoke(SessionContainer.java:648)

  at org.jboss.ejb.Container.invoke(Container.java:960)

  at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source)

  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

  at java.lang.reflect.Method.invoke(Method.java:597)

  at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)

  at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)

  at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)

  at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)

  at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)

  at org.jboss.invocation.jrmp.server.JRMPInvoker$MBeanServerAction.invoke(JRMPInvoker.java:818)

  at org.jboss.invocation.jrmp.server.JRMPInvoker.invoke(JRMPInvoker.java:419)

  at sun.reflect.GeneratedMethodAccessor164.invoke(Unknown Source)

  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

  at java.lang.reflect.Method.invoke(Method.java:597)

  at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)

  at sun.rmi.transport.Transport$1.run(Transport.java:159)

  at java.security.AccessController.doPrivileged(Native Method)

  at sun.rmi.transport.Transport.serviceCall(Transport.java:155)

  at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)

  at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)

  at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)

  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

  at java.lang.Thread.run(Thread.java:619)

-----------------------------------------

I changed the names to protect the innocent Smiley Happy Port 443 is open between the vCenter running 4.1 build 491557 and we are using VCO build 4.1 733. We do have any upgrade in the works to vCenter/VCO 5.5 but it's about a month out. Any ideas appreciated.

Tags (3)
Reply
0 Kudos
16 Replies
tschoergez
Leadership
Leadership

What version of the vcenter plugin are you using?

Can you browse the inventory in vco client down to that cluster correctly?

Joerg

Von Samsung Mobile gesendet

Reply
0 Kudos
FSvcoe
Enthusiast
Enthusiast

Untitled.png

Hi Joerg,

I installed the VC plugin version 5.0.2 and set the recommended config values for memory. No luck. Here's a screen shot of the error while trying to browse:

Also seeing the cert related errors in the log:

{http://xml.apache.org/axis/}hostname:JWQPVCO01

2014-02-10 16:55:15.885-0500 ERROR [STDERR] javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: Certificate is not in JSSECA store.

2014-02-10 16:55:15.900-0500 ERROR [STDERR]at org.apache.axis.AxisFault.makeFault(AxisFault.java:101)
Reply
0 Kudos
tschoergez
Leadership
Leadership

Doublecheck the vcenter plugin settings and make sure that the user used to connect to vcenter has the proper permissions. Last time I saw tha strange behaviour was when the admin team changed permissions in vcenter...

Gesendet von meinem LG Mobile

Reply
0 Kudos
FSvcoe
Enthusiast
Enthusiast

Hi there, checked that, and tried another user account, no luck. I just tried it again, got the error above, with the following log entries in the server.log:

-----------------------------------------------------

at java.lang.Thread.run(Thread.java:619)

2014-02-10 18:01:33.502-0500 WARN  [SDKFinder] convertToResult() --> Finder 'VC:SdkConnection' : unable to invoke read method : 'instanceUuid'

2014-02-10 18:01:33.502-0500 WARN  [SDKFinder] convertToResult() --> Finder 'VC:SdkConnection' : unexpected error 'ch.dunes.model.sdk.SDKFinderException: convertToResult() --> Finder 'VC:SdkConnection' : unable to invoke read method : 'instanceUuid''

ch.dunes.model.sdk.SDKFinderException: convertToResult() --> Finder 'VC:SdkConnection' : unable to invoke read method : 'instanceUuid'

at ch.dunes.vso.sdk.SDKFinder.convertToResult(SDKFinder.java:410)
at ch.dunes.vso.sdk.SDKFinder._findRelation(SDKFinder.java:283)
at ch.dunes.vso.sdk.SDKFinder.findRelation(SDKFinder.java:192)
at ch.dunes.vso.sdk.ModulesFactory.findRelation(ModulesFactory.java:456)
at ch.dunes.vso.sdk.mbean.ScriptingSDK.findRelation(ScriptingSDK.java:107)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
at ch.dunes.model.ejb.server.helper.SDKModulesHelper.invokeMBean(SDKModulesHelper.java:37)
at ch.dunes.model.ejb.server.helper.SDKModulesHelper.findRelation(SDKModulesHelper.java:88)
at ch.dunes.model.ejb.server.sessions.VSOFactoryBean.findRelation(VSOFactoryBean.java:1061)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.jboss.invocation.Invocation.performCall(Invocation.java:359)
at org.jboss.ejb.StatefulSessionContainer$ContainerInterceptor.invoke(StatefulSessionContainer.java:598)
at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:168)
at org.jboss.resource.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:158)
at org.jboss.ejb.plugins.CallValidationInterceptor.invoke(CallValidationInterceptor.java:63)
at org.jboss.ejb.plugins.AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:121)
at org.jboss.ejb.plugins.AbstractTxInterceptorBMT.invokeNext(AbstractTxInterceptorBMT.java:173)
at org.jboss.ejb.plugins.TxInterceptorBMT.invoke(TxInterceptorBMT.java:77)
at org.jboss.ejb.plugins.StatefulSessionInstanceInterceptor.invoke(StatefulSessionInstanceInterceptor.java:333)
at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:205)
at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:138)
at org.jboss.ejb.SessionContainer.internalInvoke(SessionContainer.java:648)
at org.jboss.ejb.Container.invoke(Container.java:960)
Reply
0 Kudos
tschoergez
Leadership
Leadership

Strange. Does removing and re-adding the vcenter host registration completely (restart the service in between).

And just to doublecheck: on bventer side everything is fine? Maybe try to restaet the vcenter webservice service on th vc box.

Gesendet von meinem LG Mobile

Reply
0 Kudos
FSvcoe
Enthusiast
Enthusiast

I've tried that as well. This was all working until early last week, been scratching my head on why might changed, I can't isolate anything different Smiley Sad

Reply
0 Kudos
tschoergez
Leadership
Leadership

Any changes in the ssl encryption on vc side? MS Update that broke stuff?

And doublecheck name resolution and ip conecction between bco and vc (but errors here usually result in a different error message)

If nothing helps, open a SR to get a webex session with our support guys...

Gesendet von meinem LG Mobile

Reply
0 Kudos
FSvcoe
Enthusiast
Enthusiast

Nothing that I know of. Guess we'll have to open a ticket. Assuming someone can help, I'll post up what I find. Thx

Reply
0 Kudos
mcfadyenj
Hot Shot
Hot Shot

I dont suppose you have installed the multi-node plugin recently.

this looks remarkably similar to issues I had in the past, the multi-node plugin seems to affect permissions as it duplicates workflows and the duplicated content does not always get accessed the same way as the internal workflows. I never bothed to dig deeper, but I found disabling multi-node fixed my issue.

not much good if you need that functionality. worth taking a look at though.


Reply
0 Kudos
FSvcoe
Enthusiast
Enthusiast

Thanks, no, no changes in the VCO config. Still working thru this prior to opening an SR.

Reply
0 Kudos
FSvcoe
Enthusiast
Enthusiast

So there's a little more info...even though the entire config is listed green, the service now won't start. Upon viewing the server.log file, we see the following:

--------------------------------------------------

erChecker] **************************************************************************************

2014-02-13 10:13:44.171-0500 FATAL [ServerChecker] **  An essential server component is missing or invalid, the server will be shutdown

2014-02-13 10:13:44.171-0500 FATAL [ServerChecker] **  Server has no valid license.

2014-02-13 10:13:44.171-0500 FATAL [ServerChecker] **************************************************************************************

2014-02-13 10:13:44.171-0500 INFO  [Server] Runtime shutdown hook called, forceHalt: true

2014-02-13 10:13:44.171-0500 INFO  [Server] JBoss SHUTDOWN: Undeploying all packages

2014-02-13 10:13:44.187-0500 INFO  [EJBDeployer] Undeploying: file:/D:/Program Files/VMware/Orchestrator/app-server/server/vmo/tmp/deploy/tmp7824318152139271972o11n-licensecheckerws-server.ear-contents/o11n-licensecheckerws-ejb-4.1.1.jar

2014-02-13 10:13:44.203-0500 INFO  [ProxyFactory] Unbind EJB Home 'LicenseCheckerWs' from jndi 'vco/LicenseCheckerWs'

2014-02-13 10:13:44.203-0500 INFO  [EjbModule] Undeployed LicenseCheckerWs

-----------------------------------------------------------------

I guess I don't understand how we could have everything report green, be able to successfully add a vCenter server and test the connection, yet get this error? Thanks,

Reply
0 Kudos
tschoergez
Leadership
Leadership

Can you try to include the vCenter license key manually to the licensing configuration tab of vCO?

Cheers,

Joerg

Reply
0 Kudos
FSvcoe
Enthusiast
Enthusiast

I can try that, but I'm concerned that somehow we have to do this. This server has been up and running for several years now, and these problems started last week, so that tells me we might somehow be masking another issue, especially with everything reporting good.

Reply
0 Kudos
tschoergez
Leadership
Leadership

strange. It might be just a situation where a strange error shows but is caused by something complete different root cause.

Can you double-check the “health” of the system (other errors in the log, disk space, database availability, recent upgrades/patches, …)?

Reply
0 Kudos
mcfadyenj
Hot Shot
Hot Shot

you mentioned the server had been running for years now.

you cert hasn't expired on the vco server has it?

Reply
0 Kudos
FSvcoe
Enthusiast
Enthusiast

Thanks. I suspect it is within this particular vCenter, as I've tried to connect from another VCO server that is working just fine, same issue. Not going to worry about it, as we're migrating all the hosts over to a brand new 5.5 VC within a few weeks anyway, so will focus on that one.

Reply
0 Kudos