Just updated my home lab vra/vro from 7.3.1 to 7.4.0 . Installation seemed to go flawlessly, except vro.
When I try to run a workflow, the workflow completes, but nothing is in the log. When I look at the system log, I see:
2018-04-12 20:28:29.332-0500 [http-nio-127.0.0.1-8280-exec-6] WARN {} [StubbornRetrier] Retry 3/6 failed because of: Could not load codec "Lucene60". Did you forget to add lucene-backward-codecs.jar?
2018-04-12 20:28:29.336-0500 [http-nio-127.0.0.1-8280-exec-9] WARN {} [FileLoggingServiceImpl] Unable to retrieve previous logs:
java.lang.RuntimeException: Cannot succeed after 6 retries.
at ch.dunes.util.StubbornRetrier.retryAtAllCosts(StubbornRetrier.java:49)
at com.vmware.o11n.service.logging.FileLoggingServiceImpl.getLogs(FileLoggingServiceImpl.java:122)
at com.vmware.o11n.service.logging.FileLoggingServiceImpl.getLogsBefore(FileLoggingServiceImpl.java:106)
at com.vmware.o11n.service.factory.VcoFactoryFacade.getLogsBefore(VcoFactoryFacade.java:3138)
at sun.reflect.GeneratedMethodAccessor246.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.vmware.o11n.service.security.AccessRightsInterceptor.invoke(AccessRightsInterceptor.java:98)
at com.vmware.o11n.service.security.AccessRightsInterceptor.invoke(AccessRightsInterceptor.java:89)
at com.vmware.o11n.service.webremoting.VcoDelegatingWebFacade.invokeOperation(VcoDelegatingWebFacade.java:105)
at com.vmware.o11n.integration.initialization.VcoFactoryServiceFacadeProxy.invokeOperation(VcoFactoryServiceFacadeProxy.java:86)
at sun.reflect.GeneratedMethodAccessor241.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy24.invokeOperation(Unknown Source)
at sun.reflect.GeneratedMethodAccessor240.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.remoting.support.RemoteInvocation.invoke(RemoteInvocation.java:212)
at org.springframework.remoting.support.DefaultRemoteInvocationExecutor.invoke(DefaultRemoteInvocationExecutor.java:39)
at org.springframework.remoting.support.RemoteInvocationBasedExporter.invoke(RemoteInvocationBasedExporter.java:78)
at org.springframework.remoting.support.RemoteInvocationBasedExporter.invokeAndCreateResult(RemoteInvocationBasedExporter.java:114)
at org.springframework.remoting.httpinvoker.HttpInvokerServiceExporter.handleRequest(HttpInvokerServiceExporter.java:80)
at org.springframework.web.context.support.HttpRequestHandlerServlet.service(HttpRequestHandlerServlet.java:67)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:215)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at com.vmware.o11n.web.auth.http.TokenAuthenticationFilter.doFilter(TokenAuthenticationFilter.java:67)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:64)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.access.channel.ChannelProcessingFilter.doFilter(ChannelProcessingFilter.java:157)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.vmware.o11n.service.spring.bootstrap.SecureSerializationFilter.doFilter(SecureSerializationFilter.java:30)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at net.sf.ehcache.constructs.web.filter.GzipFilter.doFilter(GzipFilter.java:95)
at net.sf.ehcache.constructs.web.filter.Filter.doFilter(Filter.java:86)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.vmware.o11n.service.spring.bootstrap.WebRemotingActiveNodeFilter.doFilter(WebRemotingActiveNodeFilter.java:38)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:613)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:677)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:650)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Could not load codec "Lucene60". Did you forget to add lucene-backward-codecs.jar?
at org.apache.lucene.index.SegmentInfos.readCodec(SegmentInfos.java:436)
at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:356)
at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:57)
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:54)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:77)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
at com.vmware.o11n.service.logging.FileLoggingServiceImpl.doSearchLogs(FileLoggingServiceImpl.java:312)
at com.vmware.o11n.service.logging.FileLoggingServiceImpl.findLogs(FileLoggingServiceImpl.java:195)
at com.vmware.o11n.service.logging.FileLoggingServiceImpl.access$000(FileLoggingServiceImpl.java:54)
at com.vmware.o11n.service.logging.FileLoggingServiceImpl$2.call(FileLoggingServiceImpl.java:127)
at com.vmware.o11n.service.logging.FileLoggingServiceImpl$2.call(FileLoggingServiceImpl.java:123)
at ch.dunes.util.StubbornRetrier.retryAtAllCosts(StubbornRetrier.java:27)
... 90 more
Caused by: java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.Codec with name "Lucene60" does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [Lucene62]
at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:116)
at org.apache.lucene.codecs.Codec.forName(Codec.java:116)
at org.apache.lucene.index.SegmentInfos.readCodec(SegmentInfos.java:424)
... 103 more
2018-04-12 20:28:29.338-0500 [http-nio-127.0.0.1-8280-exec-8] WARN {} [StubbornRetrier] Retry 5/6 failed because of: Could not load codec "Lucene60". Did you forget to add lucene-backward-codecs.jar?
Any thoughts?
You can try, although I doubt it will make a difference.
The interesting part the file listing is this part of Lucene index file/folder - /storage/log/vmware/vco/app-server/scripting.log_lucene.1
It seems that the old Lucene index grew so large that in addition to /storage/log/vmware/vco/app-server/scripting.log_lucene it also created /storage/log/vmware/vco/app-server/scripting.log_lucene.1
This part of the old index also need to be deleted, following the same steps I provided earlier to stop vRO service, delete the index, and start vRO service.
I'm also seeing in Catalina.out
log4j:ERROR Attempted to append to closed appender named [FILE_INTEGRATION].
I have not modified my log4j.xml
I suppose something did not update properly. To verify this, could you provide full listing of the files (after the upgrade) in the follwoing folders
/etc/vco/app-server/
/usr/lib/vco/app-server/deploy/vco/WEB-INF/lib/
What I'm looking for there is for any susspicious files with 0 bytes size and/or having an extension like .rpmsave/.rpmnew
Also, when you say 'nothing is in the log', do you mean the workflow you executed was supposed to produce some log statemenets that you don't see in the server.log? Or that there are no traces at all in server.log that the workflow has been executed at all?
vra7:~ # ls -lrt /etc/vco/app-server
total 244
-rw------- 1 vco vco 40 Aug 14 2017 keystore.password
-rw-r----- 1 vco vco 36 Aug 14 2017 node-id
-rw-r----- 1 vco vco 272 Aug 14 2017 j.stamp
-rw-r--r-- 1 vco vco 36 Aug 14 2017 vco-registration-id
-rw------- 1 vco vco 51 Mar 23 00:12 tomcat-users.xml
-rw------- 1 vco vco 67 Mar 23 00:12 jmxremote.password
-rw------- 1 vco vco 32 Mar 23 00:12 jmxremote.access
-rw------- 1 vco vco 127069 Mar 23 00:12 web.xml
-rw-r--r-- 1 vco vco 206 Mar 23 00:12 remote_facade_serialization_blacklist.txt
-rw------- 1 vco vco 1738 Mar 23 00:12 logging.properties
-rw------- 1 vco vco 1330 Mar 23 00:12 context.xml
-rw------- 1 vco vco 6521 Mar 23 00:12 catalina.properties
-rw------- 1 vco vco 0 Mar 23 00:12 catalina.policy
drwxr-xr-x 3 vco vco 4096 Mar 23 00:12 Vco
drwxr-xr-x 3 vco vco 4096 Mar 23 00:12 Catalina
-rw------- 1 vco vco 2276 Mar 23 00:13 server.xml
-rw-r--r-- 1 vco vco 0 Apr 12 18:35 update
drwxr-xr-x 2 vco vco 4096 Apr 12 18:35 security
drwxr-xr-x 2 vco vco 4096 Apr 12 18:35 ldif
drwxr-xr-x 2 vco vco 4096 Apr 12 18:35 caches
-rw------- 1 vco vco 9 Apr 12 18:35 tomcat.version
drwxr-xr-x 2 vco vco 4096 Apr 12 18:50 plugins
-rw-r--r-- 1 vco vco 1187 Apr 12 21:51 vmo.properties
-rw-r----- 1 vco vco 0 Apr 12 21:51 vmo-managed.properties
-rw------- 1 vco vco 765 Apr 12 21:51 sso.properties
-rw------- 1 vco vco 627 Apr 12 21:51 ldap.properties
-rw------- 1 vco vco 92 Apr 12 21:51 js-io-rights.conf
-rw------- 1 vco vco 426 Apr 12 21:51 cafe.properties
-rw------- 1 vco vco 10696 Apr 12 22:10 log4j.xml
vra7:~ # ls -lrt /usr/lib/vco/app-server/deploy/vco/WEB-INF/lib
total 82592
-rw-r--r-- 1 vco vco 23455 Mar 20 07:16 orchestrator-framework-7.4.0-SNAPSHOT.jar
-rw-r--r-- 1 vco vco 60573 Mar 22 16:23 vlsi-emitter-vmodl-1.0.0.jar
-rw-r--r-- 1 vco vco 514177 Mar 22 16:23 vlsi-emitter-1.0.0.jar
-rw-r--r-- 1 vco vco 250461 Mar 22 16:23 vlsi-core-1.0.0.jar
-rw-r--r-- 1 vco vco 166903 Mar 22 16:23 vlsi-client-1.0.0.jar
-rw-r--r-- 1 vco vco 39318 Mar 22 16:23 reflect-vmodl-1.0.0.jar
-rw-r--r-- 1 vco vco 42685 Mar 22 16:23 query-vmodl-1.0.0.jar
-rw-r--r-- 1 vco vco 39392 Mar 22 16:23 httpSsoAuth-1.0.0.jar
-rw-r--r-- 1 vco vco 8994 Mar 22 16:23 sso-client-1.0.0.jar
-rw-r--r-- 1 vco vco 380982 Mar 22 16:23 query-core-1.0.0.jar
-rw-r--r-- 1 vco vco 325667 Mar 22 16:23 query-client-1.0.0.jar
-rw-r--r-- 1 vco vco 85148 Mar 22 16:23 license-common-1.0.0.jar
-rw-r--r-- 1 vco vco 28642 Mar 22 16:23 license-client-1.0.0.jar
-rw-r--r-- 1 vco vco 210726 Mar 22 16:23 license-cis-vmodl-bindings-1.0.0.jar
-rw-r--r-- 1 vco vco 120769 Mar 22 16:23 invsvc-client-vmodl-1.0.0.jar
-rw-r--r-- 1 vco vco 191559 Mar 22 16:23 wstClient-1.0.0.jar
-rw-r--r-- 1 vco vco 95572 Mar 22 16:23 websso-1.0.0.jar
-rw-r--r-- 1 vco vco 172726 Mar 22 16:23 sso-adminserver-client-bindings-1.0.0.jar
-rw-r--r-- 1 vco vco 265585 Mar 22 16:23 sso-adminsdk-1.0.0.jar
-rw-r--r-- 1 vco vco 158020 Mar 22 16:23 admin-interfaces-1.0.0.jar
-rw-r--r-- 1 vco vco 5869849 Mar 22 16:24 vimVmodl-1.0.0.jar
-rw-r--r-- 1 vco vco 66965 Mar 22 16:24 vimVersions-1.0.0.jar
-rw-r--r-- 1 vco vco 40321 Mar 22 16:24 cis-common-1.0.0.jar
-rw-r--r-- 1 vco vco 37516 Mar 22 16:24 classworlds-1.1-alpha-2.jar
-rw-r--r-- 1 vco vco 3452683 Mar 22 16:24 cafe-sdk-7.4.0.jar
-rw-r--r-- 1 vco vco 153253 Mar 22 16:24 jdom-1.0.jar
-rw-r--r-- 1 vco vco 313898 Mar 22 16:24 dom4j-1.6.1.jar
-rw-r--r-- 1 vco vco 208700 Mar 22 16:25 commons-io-2.5.jar
-rw-r--r-- 1 vco vco 60686 Mar 22 16:25 commons-logging-1.1.1.jar
-rw-r--r-- 1 vco vco 305001 Mar 22 16:25 commons-httpclient-3.1.jar
-rw-r--r-- 1 vco vco 28352 Mar 22 16:25 jboss-transaction-api_1.2_spec-1.0.1.Final.jar
-rw-r--r-- 1 vco vco 66802 Mar 22 16:25 jboss-logging-3.3.0.Final.jar
-rw-r--r-- 1 vco vco 750581 Mar 22 16:25 javassist-3.20.0-GA.jar
-rw-r--r-- 1 vco vco 186741 Mar 22 16:25 jandex-2.0.3.Final.jar
-rw-r--r-- 1 vco vco 113371 Mar 22 16:25 hibernate-jpa-2.1-api-1.0.0.Final.jar
-rw-r--r-- 1 vco vco 75288 Mar 22 16:25 hibernate-commons-annotations-5.0.1.Final.jar
-rw-r--r-- 1 vco vco 284220 Mar 22 16:25 commons-lang-2.6.jar
-rw-r--r-- 1 vco vco 64110 Mar 22 16:25 classmate-1.3.0.jar
-rw-r--r-- 1 vco vco 445288 Mar 22 16:25 antlr-2.7.7.jar
-rw-r--r-- 1 vco vco 6618716 Mar 22 16:25 hibernate-core-5.2.10.Final.jar
-rw-r--r-- 1 vco vco 1129564 Mar 22 16:25 spring-core-4.3.14.RELEASE.jar
-rw-r--r-- 1 vco vco 763286 Mar 22 16:25 spring-beans-4.3.14.RELEASE.jar
-rw-r--r-- 1 vco vco 182011 Mar 22 16:25 o11n-dbinstaller-7.4.0.jar
-rw-r--r-- 1 vco vco 362679 Mar 22 16:25 commons-configuration-1.10.jar
-rw-r--r-- 1 vco vco 264226 Mar 22 16:25 spring-expression-4.3.14.RELEASE.jar
-rw-r--r-- 1 vco vco 43076 Mar 22 16:25 o11n-settings-7.4.0.jar
-rw-r--r-- 1 vco vco 486013 Mar 22 16:25 woodstox-core-asl-4.4.1.jar
-rw-r--r-- 1 vco vco 161867 Mar 22 16:25 stax2-api-3.1.4.jar
-rw-r--r-- 1 vco vco 23346 Mar 22 16:25 stax-api-1.0-2.jar
-rw-r--r-- 1 vco vco 167958 Mar 22 16:25 ognl-2.6.9.jar
-rw-r--r-- 1 vco vco 76350 Mar 22 16:25 o11n-app-settings-7.4.0.jar
-rw-r--r-- 1 vco vco 2775137 Mar 22 16:25 lucene-core-6.5.0.jar
-rw-r--r-- 1 vco vco 1513865 Mar 22 16:25 lucene-analyzers-common-6.5.0.jar
-rw-r--r-- 1 vco vco 104924 Mar 22 16:25 jchronic-0.2.6.jar
-rw-r--r-- 1 vco vco 34610 Mar 22 16:25 jackson-module-jaxb-annotations-2.8.10.jar
-rw-r--r-- 1 vco vco 1242948 Mar 22 16:25 jackson-databind-2.8.10.jar
-rw-r--r-- 1 vco vco 282634 Mar 22 16:25 jackson-core-2.8.10.jar
-rw-r--r-- 1 vco vco 55784 Mar 22 16:25 jackson-annotations-2.8.0.jar
-rw-r--r-- 1 vco vco 325500 Mar 22 16:25 httpcore-4.4.9.jar
-rw-r--r-- 1 vco vco 766123 Mar 22 16:25 httpclient-4.5.5.jar
-rw-r--r-- 1 vco vco 1338129 Mar 22 16:25 ehcache-core-2.6.2.jar
-rw-r--r-- 1 vco vco 415578 Mar 22 16:25 commons-vfs2-2.0.jar
-rw-r--r-- 1 vco vco 284184 Mar 22 16:25 commons-codec-1.10.jar
-rw-r--r-- 1 vco vco 324238 Mar 22 16:25 cglib-nodep-2.1_3.jar
-rw-r--r-- 1 vco vco 3448507 Mar 22 16:25 bcprov-jdk15on-1.56.jar
-rw-r--r-- 1 vco vco 685403 Mar 22 16:25 bcpkix-jdk15on-1.56.jar
-rw-r--r-- 1 vco vco 267761 Mar 22 16:26 spring-tx-4.3.14.RELEASE.jar
-rw-r--r-- 1 vco vco 399042 Mar 22 16:26 spring-security-core-4.2.4.RELEASE.jar
-rw-r--r-- 1 vco vco 427812 Mar 22 16:26 spring-jdbc-4.3.14.RELEASE.jar
-rw-r--r-- 1 vco vco 1141958 Mar 22 16:26 spring-context-4.3.14.RELEASE.jar
-rw-r--r-- 1 vco vco 380409 Mar 22 16:26 spring-aop-4.3.14.RELEASE.jar
-rw-r--r-- 1 vco vco 366883 Mar 22 16:26 o11n-util-7.4.0.jar
-rw-r--r-- 1 vco vco 95806 Mar 22 16:26 javax.servlet-api-3.1.0.jar
-rw-r--r-- 1 vco vco 4467 Mar 22 16:26 aopalliance-1.0.jar
-rw-r--r-- 1 vco vco 69345 Mar 22 16:26 o11n-security-7.4.0.jar
-rw-r--r-- 1 vco vco 826166 Mar 22 16:26 spring-web-4.3.14.RELEASE.jar
-rw-r--r-- 1 vco vco 265929 Mar 22 16:26 plexus-utils-1.5.7.jar
-rw-r--r-- 1 vco vco 195436 Mar 22 16:26 plexus-container-default-1.0-alpha-8.jar
-rw-r--r-- 1 vco vco 142037 Mar 22 16:26 plexus-archiver-1.0-alpha-7.jar
-rw-r--r-- 1 vco vco 4692 Mar 22 16:26 o11n-sdkapi-vmware-7.4.0.jar
-rw-r--r-- 1 vco vco 1112553 Mar 22 16:27 o11n-model-7.4.0.jar
-rw-r--r-- 1 vco vco 15680 Mar 22 16:27 o11n-j2eeutil-7.4.0.jar
-rw-r--r-- 1 vco vco 588337 Mar 22 16:27 commons-collections-3.2.2.jar
-rw-r--r-- 1 vco vco 31881 Mar 22 16:27 image4j-0.7.jar
-rw-r--r-- 1 vco vco 456206 Mar 22 16:27 o11n-gui-resources-7.4.0.jar
-rw-r--r-- 1 vco vco 530758 Mar 22 16:27 jgraphx-7.0.0.0.jar
-rw-r--r-- 1 vco vco 2666695 Mar 22 16:27 xmlbeans-2.3.0.jar
-rw-r--r-- 1 vco vco 1134765 Mar 22 16:27 rhino-1.7R4.jar
-rw-r--r-- 1 vco vco 116968 Mar 22 16:27 o11n-workflow-diagram-7.4.0.jar
-rw-r--r-- 1 vco vco 225925 Mar 22 16:27 o11n-scriptingmodel-7.4.0.jar
-rw-r--r-- 1 vco vco 85448 Mar 22 16:27 metrics-core-3.0.2.jar
-rw-r--r-- 1 vco vco 78151 Mar 22 16:28 o11n-services-7.4.0.jar
-rw-r--r-- 1 vco vco 45522 Mar 22 16:28 o11n-scriptingapi-7.4.0.jar
-rw-r--r-- 1 vco vco 31718 Mar 22 16:28 o11n-sdkdatabase-7.4.0.jar
-rw-r--r-- 1 vco vco 69409 Mar 22 16:28 activation-1.1.1.jar
-rw-r--r-- 1 vco vco 165439 Mar 22 16:28 o11n-rest-client-stubs-7.4.0.jar
-rw-r--r-- 1 vco vco 101879 Mar 22 16:28 o11n-rest-client-services-7.4.0.jar
-rw-r--r-- 1 vco vco 477813 Mar 22 16:28 spring-orm-4.3.14.RELEASE.jar
-rw-r--r-- 1 vco vco 5376 Mar 22 16:28 o11n-msgbroker-integration-7.4.0.jar
-rw-r--r-- 1 vco vco 44794 Mar 22 16:28 o11n-configuration-editors-7.4.0.jar
-rw-r--r-- 1 vco vco 450950 Mar 22 16:28 xmlsec-1.4.5.jar
-rw-r--r-- 1 vco vco 186999 Mar 22 16:28 spring-context-support-4.3.14.RELEASE.jar
-rw-r--r-- 1 vco vco 51240 Mar 22 16:28 o11n-services-bootstrap-7.4.0.jar
-rw-r--r-- 1 vco vco 84445 Mar 22 16:28 o11n-security-core-7.4.0.jar
-rw-r--r-- 1 vco vco 2442625 Mar 22 16:28 guava-20.0.jar
-rw-r--r-- 1 vco vco 15585 Mar 22 16:28 o11n-sharedfactory-7.4.0.jar
-rw-r--r-- 1 vco vco 23263 Mar 22 16:28 o11n-messaging-7.4.0.jar
-rw-r--r-- 1 vco vco 405265 Mar 22 16:29 o11n-scriptingengine-7.4.0.jar
-rw-r--r-- 1 vco vco 316268 Mar 22 16:29 o11n-sdkcenter-7.4.0.jar
-rw-r--r-- 1 vco vco 17101 Mar 22 16:30 o11n-snapshot-api-7.4.0.jar
-rw-r--r-- 1 vco vco 63777 Mar 22 16:30 validation-api-1.1.0.Final.jar
-rw-r--r-- 1 vco vco 294097 Mar 22 16:30 spring-data-jpa-1.11.10.RELEASE.jar
-rw-r--r-- 1 vco vco 786609 Mar 22 16:30 spring-data-commons-1.13.10.RELEASE.jar
-rw-r--r-- 1 vco vco 58726 Mar 22 16:30 spring-aspects-4.3.14.RELEASE.jar
-rw-r--r-- 1 vco vco 48764 Mar 22 16:30 snappy-0.3.jar
-rw-r--r-- 1 vco vco 15756 Mar 22 16:30 metrics-servlets-3.0.2.jar
-rw-r--r-- 1 vco vco 3901 Mar 22 16:30 metrics-log4j-3.0.2.jar
-rw-r--r-- 1 vco vco 30865 Mar 22 16:30 metrics-jvm-3.0.2.jar
-rw-r--r-- 1 vco vco 15800 Mar 22 16:30 metrics-json-3.0.2.jar
-rw-r--r-- 1 vco vco 9130 Mar 22 16:30 metrics-healthchecks-3.0.2.jar
-rw-r--r-- 1 vco vco 404763 Mar 22 16:30 lucene-queryparser-6.5.0.jar
-rw-r--r-- 1 vco vco 237389 Mar 22 16:30 lucene-queries-6.5.0.jar
-rw-r--r-- 1 vco vco 16515 Mar 22 16:30 jcl-over-slf4j-1.7.25.jar
-rw-r--r-- 1 vco vco 762113 Mar 22 16:30 hibernate-validator-5.4.1.Final.jar
-rw-r--r-- 1 vco vco 231952 Mar 22 16:30 gson-2.8.0.jar
-rw-r--r-- 1 vco vco 44841 Mar 22 16:30 ehcache-web-2.0.4.jar
-rw-r--r-- 1 vco vco 1929029 Mar 22 16:30 aspectjweaver-1.8.10.jar
-rw-r--r-- 1 vco vco 118762 Mar 22 16:30 aspectjrt-1.8.10.jar
-rw-r--r-- 1 vco vco 678106 Mar 22 16:31 o11n-services-server-7.4.0.jar
-rw-r--r-- 1 vco vco 85428 Mar 22 16:31 spring-oxm-4.3.14.RELEASE.jar
-rw-r--r-- 1 vco vco 39658 Mar 22 16:31 o11n-localization-7.4.0.jar
-rw-r--r-- 1 vco vco 479881 Mar 22 16:31 commons-lang3-3.5.jar
-rw-r--r-- 1 vco vco 135823 Mar 22 16:31 swagger-models-1.5.6.jar
-rw-r--r-- 1 vco vco 16066 Mar 22 16:31 swagger-annotations-1.5.6.jar
-rw-r--r-- 1 vco vco 58937 Mar 22 16:31 springfox-swagger2-2.4.0.jar
-rw-r--r-- 1 vco vco 311736 Mar 22 16:31 springfox-swagger-ui-2.4.0.jar
-rw-r--r-- 1 vco vco 84490 Mar 22 16:31 springfox-swagger-common-2.4.0.jar
-rw-r--r-- 1 vco vco 144377 Mar 22 16:31 springfox-spring-web-2.4.0.jar
-rw-r--r-- 1 vco vco 46807 Mar 22 16:31 springfox-spi-2.4.0.jar
-rw-r--r-- 1 vco vco 69577 Mar 22 16:31 springfox-schema-2.4.0.jar
-rw-r--r-- 1 vco vco 91218 Mar 22 16:31 springfox-core-2.4.0.jar
-rw-r--r-- 1 vco vco 456309 Mar 22 16:31 spring-websocket-4.3.14.RELEASE.jar
-rw-r--r-- 1 vco vco 917854 Mar 22 16:31 spring-webmvc-4.3.14.RELEASE.jar
-rw-r--r-- 1 vco vco 387097 Mar 22 16:31 spring-security-web-4.2.4.RELEASE.jar
-rw-r--r-- 1 vco vco 107499 Mar 22 16:31 spring-security-ldap-4.2.4.RELEASE.jar
-rw-r--r-- 1 vco vco 589084 Mar 22 16:31 spring-security-config-4.2.4.RELEASE.jar
-rw-r--r-- 1 vco vco 5145 Mar 22 16:31 spring-plugin-metadata-1.2.0.RELEASE.jar
-rw-r--r-- 1 vco vco 19896 Mar 22 16:31 spring-plugin-core-1.2.0.RELEASE.jar
-rw-r--r-- 1 vco vco 644934 Mar 22 16:31 mina-core-2.0.4.jar
-rw-r--r-- 1 vco vco 589803 Mar 22 16:31 xstream-1.4.10.jar
-rw-r--r-- 1 vco vco 24956 Mar 22 16:31 xpp3_min-1.1.4c.jar
-rw-r--r-- 1 vco vco 7188 Mar 22 16:31 xmlpull-1.1.3.1.jar
-rw-r--r-- 1 vco vco 7023 Mar 22 16:31 xmlbeans-xpath-2.3.0.jar
-rw-r--r-- 1 vco vco 428169 Mar 22 16:31 xmlbeans-xmlpublic-2.3.0.jar
-rw-r--r-- 1 vco vco 84091 Mar 22 16:31 xml-resolver-1.2.jar
-rw-r--r-- 1 vco vco 40153 Mar 22 16:31 taglibs-standard-spec-1.2.5.jar
-rw-r--r-- 1 vco vco 206430 Mar 22 16:31 taglibs-standard-impl-1.2.5.jar
-rw-r--r-- 1 vco vco 474522 Mar 22 16:31 spring-security-oauth2-2.0.13.RELEASE.jar
-rw-r--r-- 1 vco vco 54901 Mar 22 16:31 spring-security-crypto-4.2.4.RELEASE.jar
-rw-r--r-- 1 vco vco 378168 Mar 22 16:31 spring-ldap-core-2.3.1.RELEASE.jar
-rw-r--r-- 1 vco vco 508143 Mar 22 16:31 mail-1.4.5.jar
-rw-r--r-- 1 vco vco 4596 Mar 22 16:31 jul-to-slf4j-1.7.25.jar
-rw-r--r-- 1 vco vco 231882 Mar 22 16:31 jaxen-1.1.6.jar
-rw-r--r-- 1 vco vco 780664 Mar 22 16:31 jackson-mapper-asl-1.9.13.jar
-rw-r--r-- 1 vco vco 232248 Mar 22 16:31 jackson-core-asl-1.9.13.jar
-rw-r--r-- 1 vco vco 41753 Mar 22 16:31 httpmime-4.5.5.jar
-rw-r--r-- 1 vco vco 70604 Mar 22 16:31 commons-fileupload-1.3.3.jar
-rw-r--r-- 1 vco vco 573912 Mar 22 16:32 joda-time-2.2.jar
-rw-r--r-- 1 vco vco 2081456 Mar 22 16:32 o11n-sdk-rest-7.4.0.jar
-rw-r--r-- 1 vco vco 13486 Mar 22 16:32 o11n-apacheds-provider-7.4.0.jar
-rw-r--r-- 1 vco vco 4206262 Mar 22 16:32 apacheds-all-subset-1.5.7.jar
-rw-r--r-- 1 vco vco 449505 Mar 22 16:33 velocity-1.7.jar
-rw-r--r-- 1 vco vco 58093 Mar 22 16:35 o11n-licensing-service-7.4.0.jar
-rw-r--r-- 1 vco vco 672229 Mar 22 16:35 xmltooling-1.4.4.jar
-rw-r--r-- 1 vco vco 574069 Mar 22 16:35 openws-1.5.4.jar
-rw-r--r-- 1 vco vco 258278 Mar 22 16:35 not-yet-commons-ssl-0.3.9.jar
-rw-r--r-- 1 vco vco 367201 Mar 22 16:35 esapi-2.0.1.jar
-rw-r--r-- 1 vco vco 1413769 Mar 22 16:35 opensaml-2.6.4.jar
-rw-r--r-- 1 vco vco 81969 Mar 22 16:35 o11n-cafe-sdk-sso-7.4.0.jar
-rw-r--r-- 1 vco vco 60293 Mar 22 16:35 o11n-cis-integration-7.4.0.jar
-rw-r--r-- 1 vco vco 207594 Mar 22 16:35 o11n-security-sso-provider-7.4.0.jar
-rw-r--r-- 1 vco vco 19982 Mar 22 16:35 o11n-rest-client-sso-7.4.0.jar
-rw-r--r-- 1 vco vco 5572 Mar 22 16:35 o11n-rest-client-horizon-7.4.0.jar
-rw-r--r-- 1 vco vco 102291 Mar 22 16:35 bcmail-jdk15on-1.56.jar
-rw-r--r-- 1 vco vco 45464 Mar 22 16:36 o11n-cafe-provider-7.4.0.jar
-rw-r--r-- 1 vco vco 110573 Mar 22 16:36 o11n-licensing-provider-7.4.0.jar
-rw-r--r-- 1 vco vco 113658 Mar 22 16:36 o11n-snapshot-7.4.0.jar
-rw-r--r-- 1 vco vco 1434315 Mar 22 16:38 o11n-welcome-7.4.0.jar
One thing I noticed is that it looks like the last time I updated (from vra7.3 - 7.3.1) it updated this folders xml
vra7:~ # ls -lrt /etc/vco/configuration
total 292
-rw------- 1 vco vco 51 Mar 23 00:12 tomcat-users.xml
-rw------- 1 vco vco 67 Mar 23 00:12 jmxremote.password
-rw------- 1 vco vco 32 Mar 23 00:12 jmxremote.access
-rw------- 1 vco vco 127069 Mar 23 00:12 web.xml
-rw------- 1 vco vco 2480 Mar 23 00:12 logging.properties
-rw-r--r-- 1 vco vco 1525 Mar 23 00:12 log4j.xml
-rw------- 1 vco vco 1351 Mar 23 00:12 context.xml
-rw------- 1 vco vco 6442 Mar 23 00:12 catalina.properties
-rw------- 1 vco vco 11721 Mar 23 00:12 catalina.policy
drwxr-xr-x 3 vco vco 4096 Mar 23 00:12 Catalina
-rw------- 1 vco vco 1495 Mar 23 00:13 server.xml
-rw------- 1 vco vco 9 Apr 12 18:35 tomcat.version
-rw-r----- 1 vco vco 87203 Apr 12 21:49 _PluginInstallMetadata.info
-rw------- 1 vco vco 129 Apr 12 21:51 passwd.properties
-rw------- 1 vco vco 90 Apr 12 21:51 local.properties
-rw------- 1 vco vco 110 Apr 12 21:51 controlcenter.properties
Sorry, missed this question:
Also, when you say 'nothing is in the log', do you mean the workflow you executed was supposed to produce some log statements that you don't see in the server.log? Or that there are no traces at all in server.log that the workflow has been executed at all?
The workflow is simply 'Server.log("test");'. After execution, there is nothing in the workflow log. Server.log shows the "lucene" message over and over;
2018-04-12 20:26:38.336-0500 [http-nio-127.0.0.1-8280-exec-3] WARN {} [StubbornRetrier] Retry 5/6 failed because of: Could not load codec 'Lucene60'. Did you forget to add lucene-backward-codecs.jar?
Caused by: java.lang.IllegalArgumentException: Could not load codec 'Lucene60'. Did you forget to add lucene-backward-codecs.jar?
at org.apache.lucene.index.SegmentInfos.readCodec(SegmentInfos.java:436)
at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:356)
at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:57)
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:54)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:77)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
Caused by: java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene60' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [Lucene62]
at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:116)
at org.apache.lucene.codecs.Codec.forName(Codec.java:116)
at org.apache.lucene.index.SegmentInfos.readCodec(SegmentInfos.java:424)
I'm hoping that this is a localized issue and not an issue with the distribution. I have customers telling me that they saw that 7.4 is out.
I updated using the update iso. I can post the vami update logs if needed.
OK, the file listinglooks fine; I don't see any signs of partially/non-updated files. One file I'm not sure about is 'orchestrator-framework-7.4.0-SNAPSHOT.jar'. I don't see such file in my vRO 7.4 environment, so I'll need to check where does it come from.
Another thing that is likely to cause such codec-related error messages - you already have vRO 7.3.1 log files indexed with the old Lucene codec, and when you upgrade, the index is likely not automatically re-coded with the new Lucene codec. To check if this is indeed the case, we may try to delete the index created in 7.3.1; it will be recreated again automatically with 7.4, and there will be no loss of log content.
To delete the old Lucene index, run the following commands in a SSH session terminal
1) Stop vRO server service - service vco-server stop
2) Delete the old index - rm -rf /var/log/vco/app-server/scripting.log_lucene
3) Start vRO server service - service vco-server start in a SSH session terminal
No joy.
Deleted the scripting.log_lucene and still get the same message over and over in the log. BTW, that is the only thing being written to catalina.out.
log4j:ERROR Attempted to append to closed appender named [FILE_INTEGRATION].
/var/log/vco/app-server/scripting.log_lucene did not get re-created when I restarted vco-server.
FILE_INTEGRATION is a log appender that creates log files in format suitable to be used by Log Insight. So if you don't use and don't plan to use Log Insight in your home lab, I think you can safely remove this appenders and references to it from /etc/vco/app-server/log4j.xml. This should get rid of these annoying error messages.
For the codec error - could you run the following command to verify what lucene-related files/folders do you have in the appliance?
find / -name *lucene*
My concern is not so much the error, but the lack of logs in the designer. I've totally rebooted the appliance, and the scripting.log_lucene has not been recreated.
206 files.
vra7:~ # find / -name *lucene*
/var/lib/vco/app-server/temp/dars/o11nplugin-vsphere.dar/lib/lucene-core-6.5.0.jar
/var/lib/vco/app-server/temp/dars/o11nplugin-vsphere.dar/lib/lucene-analyzers-common-6.5.0.jar
/var/lib/codestream/plugins/fms-vrcs-ui-common-1.8.0-1241/ui/js/lib/ace-custom/mode-lucene.js
/var/lib/vrhb/vra-tests-host/sandbox/8092/lucene
/var/lib/vrhb/service-host/sandbox/8090/lucene
/usr/lib/vco/configuration/webapps/vco-controlcenter/WEB-INF/lib/lucene-queryparser-6.5.0.jar
/usr/lib/vco/configuration/webapps/vco-controlcenter/WEB-INF/lib/lucene-core-6.5.0.jar
/usr/lib/vco/configuration/webapps/vco-controlcenter/WEB-INF/lib/lucene-analyzers-common-6.5.0.jar
/usr/lib/vco/configuration/webapps/vco-controlcenter/WEB-INF/lib/lucene-queries-6.5.0.jar
/usr/lib/vco/app-server/deploy/vco/WEB-INF/lib/lucene-queryparser-6.5.0.jar
/usr/lib/vco/app-server/deploy/vco/WEB-INF/lib/lucene-core-6.5.0.jar
/usr/lib/vco/app-server/deploy/vco/WEB-INF/lib/lucene-analyzers-common-6.5.0.jar
/usr/lib/vco/app-server/deploy/vco/WEB-INF/lib/lucene-queries-6.5.0.jar
/usr/lib/vcac/tools/config/repo/lucene-queryparser-4.6.0.jar
/usr/lib/vcac/tools/config/repo/lucene-sandbox-4.6.0.jar
/usr/lib/vcac/tools/config/repo/lucene-core-4.6.0.jar
/usr/lib/vcac/tools/config/repo/lucene-queries-4.6.0.jar
/usr/lib/vcac/tools/config/repo/lucene-analyzers-common-4.6.0.jar
/usr/lib/vcac/server/webapps/ipam-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/ipam-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/ipam-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/ipam-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/ipam-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/forms-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/forms-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/forms-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/forms-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/forms-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/notification-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/notification-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/notification-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/notification-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/notification-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/management-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/management-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/management-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/management-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/management-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/network-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/network-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/network-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/network-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/network-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/advanced-designer-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/advanced-designer-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/advanced-designer-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/advanced-designer-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/advanced-designer-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/config-management-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/config-management-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/config-management-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/config-management-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/config-management-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/software-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/software-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/software-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/software-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/software-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/branding-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/branding-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/branding-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/branding-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/branding-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/endpoint-configuration-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/endpoint-configuration-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/endpoint-configuration-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/endpoint-configuration-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/endpoint-configuration-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/catalog-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/catalog-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/catalog-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/catalog-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/catalog-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/content-management-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/content-management-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/content-management-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/content-management-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/content-management-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/workitem-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/workitem-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/workitem-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/workitem-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/workitem-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/container-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar
/usr/lib/vcac/server/webapps/container-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar
/usr/lib/vcac/server/webapps/container-service/WEB-INF/lib/lucene-core-4.6.0.jar
/usr/lib/vcac/server/webapps/container-service/WEB-INF/lib/lucene-queries-4.6.0.jar
/usr/lib/vcac/server/webapps/container-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar
/usr/lib/vcac/server/webapps/properties-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/properties-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/properties-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/properties-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/properties-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/placement-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/placement-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/placement-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/placement-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/placement-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/composition-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/composition-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/composition-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/composition-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/composition-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/iaas-proxy-provider/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/iaas-proxy-provider/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/iaas-proxy-provider/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/iaas-proxy-provider/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/iaas-proxy-provider/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/approval-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/approval-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/approval-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/approval-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/approval-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/release-management-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar
/usr/lib/vcac/server/webapps/release-management-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar
/usr/lib/vcac/server/webapps/release-management-service/WEB-INF/lib/lucene-core-4.6.0.jar
/usr/lib/vcac/server/webapps/release-management-service/WEB-INF/lib/lucene-queries-4.6.0.jar
/usr/lib/vcac/server/webapps/release-management-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar
/usr/lib/vcac/server/webapps/component-registry/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/component-registry/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/component-registry/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/component-registry/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/component-registry/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/console-proxy-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/console-proxy-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/console-proxy-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/console-proxy-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/console-proxy-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/identity/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/identity/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/identity/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/identity/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/identity/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/vcac/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/vcac/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/vcac/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/vcac/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/vcac/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/healthbroker-proxy-server/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/healthbroker-proxy-server/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/healthbroker-proxy-server/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/healthbroker-proxy-server/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/healthbroker-proxy-server/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/o11n-gateway-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/o11n-gateway-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/o11n-gateway-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/o11n-gateway-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/o11n-gateway-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/event-broker-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/event-broker-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/event-broker-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/event-broker-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/event-broker-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/portal-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/portal-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/portal-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/portal-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/portal-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/reservation-service/WEB-INF/lib/lucene-sandbox-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/reservation-service/WEB-INF/lib/lucene-core-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/reservation-service/WEB-INF/lib/lucene-queries-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/reservation-service/WEB-INF/lib/lucene-analyzers-common-4.6.0.jar.moved
/usr/lib/vcac/server/webapps/reservation-service/WEB-INF/lib/lucene-queryparser-4.6.0.jar.moved
/usr/lib/vcac/server/lib/lucene-queryparser-4.6.0.jar
/usr/lib/vcac/server/lib/lucene-queryparser-4.6.0.jar.orgloc
/usr/lib/vcac/server/lib/lucene-sandbox-4.6.0.jar
/usr/lib/vcac/server/lib/lucene-sandbox-4.6.0.jar.orgloc
/usr/lib/vcac/server/lib/lucene-core-4.6.0.jar
/usr/lib/vcac/server/lib/lucene-queries-4.6.0.jar
/usr/lib/vcac/server/lib/lucene-analyzers-common-4.6.0.jar
/usr/lib/vcac/server/lib/lucene-queries-4.6.0.jar.orgloc
/usr/lib/vcac/server/lib/lucene-core-4.6.0.jar.orgloc
/usr/lib/vcac/server/lib/lucene-analyzers-common-4.6.0.jar.orgloc
/opt/vmware/horizon/workspace/webapps/AUDIT/WEB-INF/lib/lucene-spatial-4.10.4.jar
/opt/vmware/horizon/workspace/webapps/AUDIT/WEB-INF/lib/lucene-grouping-4.10.4.jar
/opt/vmware/horizon/workspace/webapps/AUDIT/WEB-INF/lib/lucene-suggest-4.10.4.jar
/opt/vmware/horizon/workspace/webapps/AUDIT/WEB-INF/lib/lucene-memory-4.10.4.jar
/opt/vmware/horizon/workspace/webapps/AUDIT/WEB-INF/lib/lucene-join-4.10.4.jar
/opt/vmware/horizon/workspace/webapps/AUDIT/WEB-INF/lib/lucene-core-4.10.4.jar
/opt/vmware/horizon/workspace/webapps/AUDIT/WEB-INF/lib/lucene-sandbox-4.10.4.jar
/opt/vmware/horizon/workspace/webapps/AUDIT/WEB-INF/lib/lucene-misc-4.10.4.jar
/opt/vmware/horizon/workspace/webapps/AUDIT/WEB-INF/lib/lucene-highlighter-4.10.4.jar
/opt/vmware/horizon/workspace/webapps/AUDIT/WEB-INF/lib/lucene-queryparser-4.10.4.jar
/opt/vmware/horizon/workspace/webapps/AUDIT/WEB-INF/lib/lucene-queries-4.10.4.jar
/opt/vmware/horizon/workspace/webapps/AUDIT/WEB-INF/lib/lucene-analyzers-common-4.10.4.jar
/opt/vmware/elasticsearch/lib/lucene-expressions-4.10.4.jar
/opt/vmware/elasticsearch/lib/lucene-spatial-4.10.4.jar
/opt/vmware/elasticsearch/lib/lucene-grouping-4.10.4.jar
/opt/vmware/elasticsearch/lib/lucene-suggest-4.10.4.jar
/opt/vmware/elasticsearch/lib/lucene-memory-4.10.4.jar
/opt/vmware/elasticsearch/lib/lucene-join-4.10.4.jar
/opt/vmware/elasticsearch/lib/lucene-core-4.10.4.jar
/opt/vmware/elasticsearch/lib/lucene-sandbox-4.10.4.jar
/opt/vmware/elasticsearch/lib/lucene-misc-4.10.4.jar
/opt/vmware/elasticsearch/lib/lucene-highlighter-4.10.4.jar
/opt/vmware/elasticsearch/lib/lucene-queryparser-4.10.4.jar
/opt/vmware/elasticsearch/lib/lucene-queries-4.10.4.jar
/opt/vmware/elasticsearch/lib/lucene-analyzers-common-4.10.4.jar
/storage/log/vmware/vco/app-server/scripting.log_lucene.1
/etc/xenon/lib/lucene-backward-codecs-6.5.0.jar
/etc/xenon/lib/lucene-grouping-6.5.0.jar
/etc/xenon/lib/lucene-misc-6.5.0.jar
/etc/xenon/lib/lucene-core-6.5.0.jar
/etc/xenon/lib/lucene-analyzers-common-6.5.0.jar
/etc/xenon/lib/lucene-queries-6.5.0.jar
/etc/xenon/storage/8485/lucene
Do you think I should revert to pre-update snapshot and retry the upgrade?
You can try, although I doubt it will make a difference.
The interesting part the file listing is this part of Lucene index file/folder - /storage/log/vmware/vco/app-server/scripting.log_lucene.1
It seems that the old Lucene index grew so large that in addition to /storage/log/vmware/vco/app-server/scripting.log_lucene it also created /storage/log/vmware/vco/app-server/scripting.log_lucene.1
This part of the old index also need to be deleted, following the same steps I provided earlier to stop vRO service, delete the index, and start vRO service.
Stopped service, deleted /storage/log/vmware/vco/app-server/scripting.log_lucene.1, and restarted service. That fixed it...
I did not modify anything in log4j.xml as you suggested (for anyone else who may see this)
FILE_INTEGRATION is a log appender that creates log files in format suitable to be used by Log Insight. So if you don't use and don't plan to use Log Insight in your home lab, I think you can safely remove this appenders and references to it from /etc/vco/app-server/log4j.xml. This should get rid of these annoying error messages.
I really appreciate this.
So solution:
1) Stop vRO server service - service vco-server stop
2) Delete the old index - rm -rf /var/log/vco/app-server/scripting.log_lucene
3) Delete the old index - rm -rf /var/log/vco/app-server/scripting.log_lucene.1 (if exist)
4) Start vRO server service - service vco-server start in a SSH session terminal
YEP,
same problem for us & same workaround :
service vco-server stop
rm -rf /var/log/vco/app-server/scripting.log_lucene
rm -rf /var/log/vco/app-server/scripting.log_lucene.1 (if exist)
rm -rf /var/log/vco/app-server/scripting.log_lucene.2 (if exist)
rm -rf /var/log/vco/app-server/scripting.log_lucene.3 (if exist)
rm -rf /var/log/vco/app-server/scripting.log_lucene.4 (if exist)
rm -rf /var/log/vco/app-server/scripting.log_lucene.5 (if exist)
service vco-server start
thx for your help @MikeNox
Same problem here e same workaround too.
Thanks.
I just completed another upgrade. This time it had 1 integrated vro & 2 external. Again, going from 7.3.1.
The 2 externals had to have this solution applied, but the internal one was fine, so it looks like it is definitely repeatable.
service vco-server stop
rm -rf /var/log/vco/app-server/scripting.log_lucene*
service vco-server start
Thank you,
Same problem and fix upgrading from 7.3.0 to 7.4
Hi experience the exact same problem after upgrading from vRO7.2 to 7.4.
I was not getting any proper logging as well as debug logs in vRO client was not showing when i run a workflow.
Deleting the files scripting.log_lucene and scripting.log_lucene1 through 5 seems to have fix the problem for me as well.
Now however i see the following errors in the server.log file.
2018-05-25 21:00:14.107+0000 [https-jsse-nio-0.0.0.0-8281-exec-10] ERROR {} [VcoFactoryServiceFacadeProxy] ch.dunes.util.DunesServerException: could not execute query; SQL [select vmologeven0_.id as id1_19_, vmologeven0_.logtimestamp as logtimes2_19_, vmologeven0_.longdescription as longdesc3_19_, vmologeven0_.originatorid as originat4_19_, vmologeven0_.originatortype as originat5_19_, vmologeven0_.originatoruri as originat6_19_, vmologeven0_.originatorusername as originat7_19_, vmologeven0_.severity as severity8_19_, vmologeven0_.shortdescription as shortdes9_19_ from vmo_logevent vmologeven0_ where vmologeven0_.originatorid=? and vmologeven0_.originatoruri=? and (lower(vmologeven0_.shortdescription) like ? or lower(vmologeven0_.longdescription) like ?) order by vmologeven0_.logtimestamp desc]; nested exception is org.hibernate.exception.SQLGrammarException: could not execute query
2018-05-25 21:00:14.102+0000 [https-jsse-nio-0.0.0.0-8281-exec-10] WARN {} [SqlExceptionHelper] SQL Error: 0, SQLState: 07009
2018-05-25 21:00:14.102+0000 [https-jsse-nio-0.0.0.0-8281-exec-10] ERROR {} [SqlExceptionHelper] Invalid parameter index 5.
2018-05-25 21:00:14.103+0000 [https-jsse-nio-0.0.0.0-8281-exec-10] ERROR {} [VcoDelegatingWebFacade] Server Error...
Basically, this error means the code which tries to execute this SQL query passes more parameters than expected. What kind of database do you use?
Sorry for the late reponse.
I am using an external standalone SQL 2012 SP2 (11.0.5058.0).
Any idea on what might be causing this issue and how I can go about resolving it?
Thanks
Had the same problem with external 7.3 and your solution fixed it
thx a lot iiliev