VMware Cloud Community
Kanstantsin
Enthusiast
Enthusiast

vCenter 6.0

Hi all,

I am not understanding how work vCenter 6.0.

There are two sites. On site 1 installed PSC1 and vCenter1, on site2 - PSC2 vCenter2. PSC2 use existing domain PSC1. When site 2 goes down, on Web interface vCenter1 I see "Could not connect to one or more vCenter Server systems:https://vcenter2.bgs.insu:443/sdk"

vCenter1 Web interface work very bad, It retarding. I can not open "Edit Settings" on all VMs.

I thought vCenter must work correct if another vcenter in linked moge goes down. Maybe I do not know something.

vsphere_client_virgo.log

com.vmware.vise.data.query.DataServiceException: The data service timed out because a back-end task took more than 120 seconds. See the vsphere-client-virgo.log for more details.

    at com.vmware.vise.data.query.impl.DataServiceImpl.getResults(DataServiceImpl.java:416)

    at com.vmware.vise.data.query.impl.DataServiceImpl.getData(DataServiceImpl.java:220)

    at sun.reflect.GeneratedMethodAccessor286.invoke(Unknown Source)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke(Method.java:606)

    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:319)

    at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)

    at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)

    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)

    at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)

    at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)

    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)

    at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)

    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)

    at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)

    at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)

    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)

    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)

    at com.sun.proxy.$Proxy229.getData(Unknown Source)

    at sun.reflect.GeneratedMethodAccessor331.invoke(Unknown Source)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke(Method.java:606)

    at flex.messaging.services.remoting.adapters.JavaAdapter.invoke(JavaAdapter.java:418)

    at com.vmware.vise.messaging.remoting.JavaAdapterEx.invoke(JavaAdapterEx.java:50)

    at flex.messaging.services.RemotingService.serviceMessage(RemotingService.java:183)

    at flex.messaging.MessageBroker.routeMessageToService(MessageBroker.java:1400)

    at flex.messaging.endpoints.AbstractEndpoint.serviceMessage(AbstractEndpoint.java:1011)

    at flex.messaging.endpoints.AbstractEndpoint$$FastClassByCGLIB$$1a3ef066.invoke(<generated>)

    at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)

    at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689)

    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)

    at org.springframework.flex.core.MessageInterceptionAdvice.invoke(MessageInterceptionAdvice.java:66)

    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)

    at org.springframework.aop.framework.adapter.ThrowsAdviceInterceptor.invoke(ThrowsAdviceInterceptor.java:124)

    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)

    at org.springframework.aop.framework.Cglib2AopProxy$FixedChainStaticTargetInterceptor.intercept(Cglib2AopProxy.java:573)

    at com.vmware.vise.messaging.endpoints.AMFEndpoint$$EnhancerByCGLIB$$88f5b58f.serviceMessage(<generated>)

    at flex.messaging.endpoints.amf.MessageBrokerFilter.invoke(MessageBrokerFilter.java:103)

    at flex.messaging.endpoints.amf.LegacyFilter.invoke(LegacyFilter.java:158)

    at flex.messaging.endpoints.amf.SessionFilter.invoke(SessionFilter.java:44)

    at flex.messaging.endpoints.amf.BatchProcessFilter.invoke(BatchProcessFilter.java:67)

    at flex.messaging.endpoints.amf.SerializationFilter.invoke(SerializationFilter.java:166)

    at flex.messaging.endpoints.BaseHTTPEndpoint.service(BaseHTTPEndpoint.java:291)

    at com.vmware.vise.messaging.endpoints.AMFEndpoint$$EnhancerByCGLIB$$88f5b58f.service(<generated>)

    at org.springframework.flex.servlet.MessageBrokerHandlerAdapter.handle(MessageBrokerHandlerAdapter.java:109)

    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)

    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)

    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)

    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:789)

    at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)

    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)

    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)

    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)

    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)

    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)

    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)

    at com.vmware.vise.security.SessionManagementFilter.doFilterInternal(SessionManagementFilter.java:82)

    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)

    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)

    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)

    at com.vmware.vsphere.client.logging.MDCLogFilter.doFilterInternal(MDCLogFilter.java:41)

    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)

    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)

    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)

    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)

    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)

    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)

    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)

    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)

    at org.eclipse.virgo.web.tomcat.support.ApplicationNameTrackingValve.invoke(ApplicationNameTrackingValve.java:33)

    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)

    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)

    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)

    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)

    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)

    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:315)

    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

    at java.lang.Thread.run(Thread.java:745)

vapi logs

2015-07-29T10:36:08.947+03:00 | WARN  | state-manager1        | InvProviderClientFactory   | Unable to login/logout to/from inventory service

com.vmware.vim.query.client.exception.ClientException: java.util.concurrent.ExecutionException: com.vmware.vim.vmomi.client.exception.ConnectionException: java.net.ConnectException: Connection timed out: connect

    at com.vmware.vim.query.client.impl.QueryAuthenticationManagerImpl.loginBySamlToken(QueryAuthenticationManagerImpl.java:205)

    at com.vmware.vapi.endpoint.cis.router.InvProviderClientFactory.createProviderClient(InvProviderClientFactory.java:92)

    at com.vmware.vapi.endpoint.cis.router.InvSvcBuilder.createInvServiceClientList(InvSvcBuilder.java:307)

    at com.vmware.vapi.endpoint.cis.router.InvSvcBuilder.buildInt(InvSvcBuilder.java:279)

    at com.vmware.vapi.endpoint.cis.router.InvSvcBuilder.rebuild(InvSvcBuilder.java:237)

    at com.vmware.vapi.state.impl.DefaultStateManager.rebuild(DefaultStateManager.java:390)

    at com.vmware.vapi.state.impl.DefaultStateManager$2.run(DefaultStateManager.java:426)

    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)

    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)

    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

    at java.lang.Thread.run(Thread.java:745)

Caused by: java.util.concurrent.ExecutionException: com.vmware.vim.vmomi.client.exception.ConnectionException: java.net.ConnectException: Connection timed out: connect

    at com.vmware.vim.vmomi.core.impl.BlockingFuture.get(BlockingFuture.java:70)

    at com.vmware.vim.query.client.impl.QueryAuthenticationManagerImpl.loginBySamlToken(QueryAuthenticationManagerImpl.java:203)

    ... 13 more

Caused by: com.vmware.vim.vmomi.client.exception.ConnectionException: java.net.ConnectException: Connection timed out: connect

    at com.vmware.vim.vmomi.client.common.impl.ResponseImpl.setError(ResponseImpl.java:251)

    at com.vmware.vim.vmomi.client.http.impl.HttpExchange.run(HttpExchange.java:54)

    at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingBase.executeRunnable(HttpProtocolBindingBase.java:186)

    at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingImpl.send(HttpProtocolBindingImpl.java:77)

    at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl$CallExecutor.sendCall(MethodInvocationHandlerImpl.java:581)

    at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl$CallExecutor.executeCall(MethodInvocationHandlerImpl.java:562)

    at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.completeCall(MethodInvocationHandlerImpl.java:348)

    at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.invokeOperation(MethodInvocationHandlerImpl.java:308)

    at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.invoke(MethodInvocationHandlerImpl.java:182)

    at com.sun.proxy.$Proxy80.loginBySamlToken(Unknown Source)

    at com.vmware.vim.query.client.impl.QueryAuthenticationManagerImpl.loginBySamlToken(QueryAuthenticationManagerImpl.java:201)

    ... 13 more

Caused by: java.net.ConnectException: Connection timed out: connect

    at java.net.DualStackPlainSocketImpl.connect0(Native Method)

    at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)

    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)

    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)

    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)

    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)

    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

    at java.net.Socket.connect(Socket.java:579)

    at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618)

    at com.vmware.vim.vmomi.client.http.impl.VlsiSslSocketFactory.connectSocket(VlsiSslSocketFactory.java:95)

    at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:401)

    at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)

    at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:144)

    at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:131)

    at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)

    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)

    at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)

    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)

    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)

    at com.vmware.vim.vmomi.client.http.impl.HttpExchange.run(HttpExchange.java:48)

    ... 22 more

2015-07-29T10:36:08.947+03:00 | WARN  | state-manager1        | InvSvcBuilder              | The list of inventory service clients contains empty provider. Building new list of providers.
2015-07-29T10:36:08.948+03:00 | WARN  | state-manager1        | InvSvcBuilder              | Inventory service clients list contains empty provider. The federated IS query in the router might not be accurate.

Can you help me unredstand it?

How I can resolve it?

Thanks in advance.

Reply
0 Kudos
1 Reply
bondspot
Contributor
Contributor

Hi,

same configuration (PSC1, VCSA1 in first datacenter, PSC2, VCSA2 in second datacenter, PSC1 and PSC2 use same SSO domain), same error in my pre-production environment when link between datacenter is broken. I want to point out that we do not use any load balancers between VCSA and PSC.

Observed very long loggin into vCenter Web Interface located in working datacenter. Something is very broken with linked mode in vSphere 6.

regards

Jakub

Reply
0 Kudos