VMware Cloud Community
Kanstantsin
Enthusiast
Enthusiast

Vmware vapi service health alarm

I have installed two vCenter servers (with extended PSC)  in enhanced linked mode.

I can't connect to second vCenter2, but I can connect to vCenter1. In vCenter1 web client I see alarm "vAPI endpoint vCenter2. vmware vapi service health alarm".

There are some errors In C:\ProgramData\VMware\vCenterServer\logs\vapi\endpoint\endpoint.txt

2015-05-03T00:26:18.284+03:00 | WARN  | state-manager1        | InvProviderClientFactory   | Unable to login/logout to/from inventory service

com.vmware.vim.query.client.exception.ClientException: java.util.concurrent.ExecutionException: (vmodl.fault.SecurityError) {

   faultCause = null,

   faultMessage = null

}

    at com.vmware.vim.query.client.impl.QueryAuthenticationManagerImpl.loginBySamlToken(QueryAuthenticationManagerImpl.java:205)

    at com.vmware.vapi.endpoint.cis.router.InvProviderClientFactory.createProviderClient(InvProviderClientFactory.java:92)

    at com.vmware.vapi.endpoint.cis.router.InvSvcBuilder.createInvServiceClientList(InvSvcBuilder.java:307)

    at com.vmware.vapi.endpoint.cis.router.InvSvcBuilder.buildInt(InvSvcBuilder.java:279)

    at com.vmware.vapi.endpoint.cis.router.InvSvcBuilder.rebuild(InvSvcBuilder.java:237)

    at com.vmware.vapi.state.impl.DefaultStateManager.rebuild(DefaultStateManager.java:390)

    at com.vmware.vapi.state.impl.DefaultStateManager$2.run(DefaultStateManager.java:426)

    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)

    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)

    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

    at java.lang.Thread.run(Thread.java:745)

Caused by: java.util.concurrent.ExecutionException: (vmodl.fault.SecurityError) {

   faultCause = null,

   faultMessage = null

}

    at com.vmware.vim.vmomi.core.impl.BlockingFuture.get(BlockingFuture.java:70)

    at com.vmware.vim.query.client.impl.QueryAuthenticationManagerImpl.loginBySamlToken(QueryAuthenticationManagerImpl.java:203)

    ... 13 more

Caused by: (vmodl.fault.SecurityError) {

   faultCause = null,

   faultMessage = null

}

    at sun.reflect.GeneratedConstructorAccessor54595.newInstance(Unknown Source)

    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

    at java.lang.Class.newInstance(Class.java:379)

    at com.vmware.vim.vmomi.core.types.impl.ComplexTypeImpl.newInstance(ComplexTypeImpl.java:173)

    at com.vmware.vim.vmomi.core.types.impl.DefaultDataObjectFactory.newDataObject(DefaultDataObjectFactory.java:26)

    at com.vmware.vim.vmomi.core.soap.impl.unmarshaller.ComplexStackContext.<init>(ComplexStackContext.java:31)

    at com.vmware.vim.vmomi.core.soap.impl.unmarshaller.UnmarshallerImpl$UnmarshallSoapFaultContext.parse(UnmarshallerImpl.java:141)

    at com.vmware.vim.vmomi.core.soap.impl.unmarshaller.UnmarshallerImpl$UnmarshallSoapFaultContext.unmarshall(UnmarshallerImpl.java:102)

    at com.vmware.vim.vmomi.core.soap.impl.unmarshaller.UnmarshallerImpl.unmarshalSoapFault(UnmarshallerImpl.java:89)

    at com.vmware.vim.vmomi.core.soap.impl.unmarshaller.UnmarshallerImpl.unmarshalSoapFault(UnmarshallerImpl.java:84)

    at com.vmware.vim.vmomi.client.common.impl.SoapFaultStackContext.setValue(SoapFaultStackContext.java:41)

    at com.vmware.vim.vmomi.client.common.impl.ResponseUnmarshaller.unmarshal(ResponseUnmarshaller.java:112)

    at com.vmware.vim.vmomi.client.common.impl.ResponseImpl.unmarshalResponse(ResponseImpl.java:273)

    at com.vmware.vim.vmomi.client.common.impl.ResponseImpl.setResponse(ResponseImpl.java:230)

    at com.vmware.vim.vmomi.client.http.impl.HttpExchangeBase.parseResponse(HttpExchangeBase.java:144)

    at com.vmware.vim.vmomi.client.http.impl.HttpExchange.run(HttpExchange.java:51)

    at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingBase.executeRunnable(HttpProtocolBindingBase.java:186)

    at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingImpl.send(HttpProtocolBindingImpl.java:77)

    at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl$CallExecutor.sendCall(MethodInvocationHandlerImpl.java:581)

    at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl$CallExecutor.executeCall(MethodInvocationHandlerImpl.java:562)

    at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.completeCall(MethodInvocationHandlerImpl.java:348)

    at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.invokeOperation(MethodInvocationHandlerImpl.java:308)

    at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.invoke(MethodInvocationHandlerImpl.java:182)

    at com.sun.proxy.$Proxy79.loginBySamlToken(Unknown Source)

    at com.vmware.vim.query.client.impl.QueryAuthenticationManagerImpl.loginBySamlToken(QueryAuthenticationManagerImpl.java:201)

    ... 13 more

2015-05-03T00:17:44.369+03:00 | ERROR | state-manager1        | RestBuilder                | Failed to create the REST.

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.vmware.vapi.rest.configuration.LinkLocatorRegistrator#0' defined in class path resource [rest-navigation.xml]: Invocation of init method failed; nested exception is java.lang.OutOfMemoryError: GC overhead limit exceeded

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1455)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)

    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)

    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)

    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)

    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)

    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585)

    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:913)

    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:464)

    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)

    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:105)

    at com.vmware.vapi.endpoint.rest.RestBuilder.createRestServlet(RestBuilder.java:140)

    at com.vmware.vapi.endpoint.rest.RestBuilder.doCreateRest(RestBuilder.java:100)

    at com.vmware.vapi.endpoint.rest.RestBuilder.rebuild(RestBuilder.java:55)

    at com.vmware.vapi.state.impl.DefaultStateManager.rebuild(DefaultStateManager.java:390)

    at com.vmware.vapi.state.impl.DefaultStateManager$2.run(DefaultStateManager.java:426)

    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)

    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)

    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

    at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

    at java.util.Arrays.copyOf(Arrays.java:2367)

    at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)

    at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)

    at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:480)

    at java.lang.StringBuffer.append(StringBuffer.java:309)

    at java.util.regex.Matcher.appendTail(Matcher.java:862)

    at java.util.regex.Matcher.replaceAll(Matcher.java:909)

    at java.lang.String.replace(String.java:2180)

    at com.vmware.vapi.rest.RestUrlUtils.buildStaticUriForService(RestUrlUtils.java:400)

    at com.vmware.vapi.rest.RestUrlUtils.buildStaticUriForService(RestUrlUtils.java:382)

    at com.vmware.vapi.rest.link.impl.RestLinkLocatorImpl.registerBaseHref(RestLinkLocatorImpl.java:135)

    at com.vmware.vapi.rest.link.impl.RestLinkLocatorImpl.register(RestLinkLocatorImpl.java:52)

    at com.vmware.vapi.rest.configuration.LinkLocatorRegistrator.registerLinkData(LinkLocatorRegistrator.java:34)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke(Method.java:606)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1581)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1522)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1452)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)

    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)

    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)

    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)

    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)

    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585)

    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:913)

    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:464)

    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)

    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:105)

    at com.vmware.vapi.endpoint.rest.RestBuilder.createRestServlet(RestBuilder.java:140)

In vsphere_client_virgo.txt

[2015-05-01T16:54:04.746+03:00] [ERROR] ing.timer.TimerFactoryBean#0  com.vmware.vise.vim.cm.healthstatus.InventoryServiceHealth    Search failed for Health Status Endpointcom.vmware.vim.vmomi.client.exception.ConnectionException: java.net.SocketException: Connection reset

[2015-05-01T16:54:04.746+03:00] [ERROR] ing.timer.TimerFactoryBean#0  com.vmware.vise.vim.cm.healthstatus.InventoryServiceHealth    Could not find Health Status EndPoint for Inventory Service

In vCenter1 I saw this error, but after some days  it disappeared.

How I can fix it?

Thanks in advance.

Reply
0 Kudos
5 Replies
Kanstantsin
Enthusiast
Enthusiast

I use vcenter 6.0

Reply
0 Kudos
GuoYue
Contributor
Contributor

I find the same error.

[2016-06-03T16:03:29.044+08:00] [ERROR] ing.timer.TimerFactoryBean#0  com.vmware.vise.vim.cm.healthstatus.InventoryServiceHealth    Could not find Health Status EndPoint for Inventory Service
Reply
0 Kudos
Orcunus
Enthusiast
Enthusiast

This might help.

vAPI Endpoint out of memory issue in vSphere 6.x federated vCenter Server environments results in pa...

You may also try to increase JVM heap size of the vAPI Endpoint service. Let me know if you need help on that.

Reply
0 Kudos
vnaseri
Contributor
Contributor

We have the same issue with vSphere 6.5!! I reset the Service but the alarm is still there don't know what to do, please help! thanks

vSphere Web Client : Version 6.5.0 Build 5178943

VMware ESXi: Version 6.5.0 Build 4887370

Reply
0 Kudos
sandykaushik
Contributor
Contributor

vAPI endpoint status changes to yellow in vCenter Server (2144715)

Regards
Sandeep Kaushik
Reply
0 Kudos