Highlighted
Enthusiast
Enthusiast

Vmware vapi service health alarm

I have installed two vCenter servers (with extended PSC)  in enhanced linked mode.

I can't connect to second vCenter2, but I can connect to vCenter1. In vCenter1 web client I see alarm "vAPI endpoint vCenter2. vmware vapi service health alarm".

There are some errors In C:\ProgramData\VMware\vCenterServer\logs\vapi\endpoint\endpoint.txt

2015-05-03T00:26:18.284+03:00 | WARN  | state-manager1        | InvProviderClientFactory   | Unable to login/logout to/from inventory service

com.vmware.vim.query.client.exception.ClientException: java.util.concurrent.ExecutionException: (vmodl.fault.SecurityError) {

   faultCause = null,

   faultMessage = null

}

    at com.vmware.vim.query.client.impl.QueryAuthenticationManagerImpl.loginBySamlToken(QueryAuthenticationManagerImpl.java:205)

    at com.vmware.vapi.endpoint.cis.router.InvProviderClientFactory.createProviderClient(InvProviderClientFactory.java:92)

    at com.vmware.vapi.endpoint.cis.router.InvSvcBuilder.createInvServiceClientList(InvSvcBuilder.java:307)

    at com.vmware.vapi.endpoint.cis.router.InvSvcBuilder.buildInt(InvSvcBuilder.java:279)

    at com.vmware.vapi.endpoint.cis.router.InvSvcBuilder.rebuild(InvSvcBuilder.java:237)

    at com.vmware.vapi.state.impl.DefaultStateManager.rebuild(DefaultStateManager.java:390)

    at com.vmware.vapi.state.impl.DefaultStateManager$2.run(DefaultStateManager.java:426)

    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)

    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)

    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

    at java.lang.Thread.run(Thread.java:745)

Caused by: java.util.concurrent.ExecutionException: (vmodl.fault.SecurityError) {

   faultCause = null,

   faultMessage = null

}

    at com.vmware.vim.vmomi.core.impl.BlockingFuture.get(BlockingFuture.java:70)

    at com.vmware.vim.query.client.impl.QueryAuthenticationManagerImpl.loginBySamlToken(QueryAuthenticationManagerImpl.java:203)

    ... 13 more

Caused by: (vmodl.fault.SecurityError) {

   faultCause = null,

   faultMessage = null

}

    at sun.reflect.GeneratedConstructorAccessor54595.newInstance(Unknown Source)

    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

    at java.lang.Class.newInstance(Class.java:379)

    at com.vmware.vim.vmomi.core.types.impl.ComplexTypeImpl.newInstance(ComplexTypeImpl.java:173)

    at com.vmware.vim.vmomi.core.types.impl.DefaultDataObjectFactory.newDataObject(DefaultDataObjectFactory.java:26)

    at com.vmware.vim.vmomi.core.soap.impl.unmarshaller.ComplexStackContext.<init>(ComplexStackContext.java:31)

    at com.vmware.vim.vmomi.core.soap.impl.unmarshaller.UnmarshallerImpl$UnmarshallSoapFaultContext.parse(UnmarshallerImpl.java:141)

    at com.vmware.vim.vmomi.core.soap.impl.unmarshaller.UnmarshallerImpl$UnmarshallSoapFaultContext.unmarshall(UnmarshallerImpl.java:102)

    at com.vmware.vim.vmomi.core.soap.impl.unmarshaller.UnmarshallerImpl.unmarshalSoapFault(UnmarshallerImpl.java:89)

    at com.vmware.vim.vmomi.core.soap.impl.unmarshaller.UnmarshallerImpl.unmarshalSoapFault(UnmarshallerImpl.java:84)

    at com.vmware.vim.vmomi.client.common.impl.SoapFaultStackContext.setValue(SoapFaultStackContext.java:41)

    at com.vmware.vim.vmomi.client.common.impl.ResponseUnmarshaller.unmarshal(ResponseUnmarshaller.java:112)

    at com.vmware.vim.vmomi.client.common.impl.ResponseImpl.unmarshalResponse(ResponseImpl.java:273)

    at com.vmware.vim.vmomi.client.common.impl.ResponseImpl.setResponse(ResponseImpl.java:230)

    at com.vmware.vim.vmomi.client.http.impl.HttpExchangeBase.parseResponse(HttpExchangeBase.java:144)

    at com.vmware.vim.vmomi.client.http.impl.HttpExchange.run(HttpExchange.java:51)

    at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingBase.executeRunnable(HttpProtocolBindingBase.java:186)

    at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingImpl.send(HttpProtocolBindingImpl.java:77)

    at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl$CallExecutor.sendCall(MethodInvocationHandlerImpl.java:581)

    at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl$CallExecutor.executeCall(MethodInvocationHandlerImpl.java:562)

    at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.completeCall(MethodInvocationHandlerImpl.java:348)

    at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.invokeOperation(MethodInvocationHandlerImpl.java:308)

    at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.invoke(MethodInvocationHandlerImpl.java:182)

    at com.sun.proxy.$Proxy79.loginBySamlToken(Unknown Source)

    at com.vmware.vim.query.client.impl.QueryAuthenticationManagerImpl.loginBySamlToken(QueryAuthenticationManagerImpl.java:201)

    ... 13 more

2015-05-03T00:17:44.369+03:00 | ERROR | state-manager1        | RestBuilder                | Failed to create the REST.

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.vmware.vapi.rest.configuration.LinkLocatorRegistrator#0' defined in class path resource [rest-navigation.xml]: Invocation of init method failed; nested exception is java.lang.OutOfMemoryError: GC overhead limit exceeded

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1455)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)

    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)

    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)

    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)

    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)

    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585)

    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:913)

    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:464)

    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)

    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:105)

    at com.vmware.vapi.endpoint.rest.RestBuilder.createRestServlet(RestBuilder.java:140)

    at com.vmware.vapi.endpoint.rest.RestBuilder.doCreateRest(RestBuilder.java:100)

    at com.vmware.vapi.endpoint.rest.RestBuilder.rebuild(RestBuilder.java:55)

    at com.vmware.vapi.state.impl.DefaultStateManager.rebuild(DefaultStateManager.java:390)

    at com.vmware.vapi.state.impl.DefaultStateManager$2.run(DefaultStateManager.java:426)

    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)

    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)

    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

    at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

    at java.util.Arrays.copyOf(Arrays.java:2367)

    at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)

    at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)

    at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:480)

    at java.lang.StringBuffer.append(StringBuffer.java:309)

    at java.util.regex.Matcher.appendTail(Matcher.java:862)

    at java.util.regex.Matcher.replaceAll(Matcher.java:909)

    at java.lang.String.replace(String.java:2180)

    at com.vmware.vapi.rest.RestUrlUtils.buildStaticUriForService(RestUrlUtils.java:400)

    at com.vmware.vapi.rest.RestUrlUtils.buildStaticUriForService(RestUrlUtils.java:382)

    at com.vmware.vapi.rest.link.impl.RestLinkLocatorImpl.registerBaseHref(RestLinkLocatorImpl.java:135)

    at com.vmware.vapi.rest.link.impl.RestLinkLocatorImpl.register(RestLinkLocatorImpl.java:52)

    at com.vmware.vapi.rest.configuration.LinkLocatorRegistrator.registerLinkData(LinkLocatorRegistrator.java:34)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke(Method.java:606)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1581)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1522)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1452)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)

    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)

    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)

    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)

    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)

    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)

    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585)

    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:913)

    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:464)

    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)

    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:105)

    at com.vmware.vapi.endpoint.rest.RestBuilder.createRestServlet(RestBuilder.java:140)

In vsphere_client_virgo.txt

[2015-05-01T16:54:04.746+03:00] [ERROR] ing.timer.TimerFactoryBean#0  com.vmware.vise.vim.cm.healthstatus.InventoryServiceHealth    Search failed for Health Status Endpointcom.vmware.vim.vmomi.client.exception.ConnectionException: java.net.SocketException: Connection reset

[2015-05-01T16:54:04.746+03:00] [ERROR] ing.timer.TimerFactoryBean#0  com.vmware.vise.vim.cm.healthstatus.InventoryServiceHealth    Could not find Health Status EndPoint for Inventory Service

In vCenter1 I saw this error, but after some days  it disappeared.

How I can fix it?

Thanks in advance.

0 Kudos
5 Replies
Highlighted
Enthusiast
Enthusiast

I use vcenter 6.0

0 Kudos
Highlighted
Contributor
Contributor

I find the same error.

[2016-06-03T16:03:29.044+08:00] [ERROR] ing.timer.TimerFactoryBean#0  com.vmware.vise.vim.cm.healthstatus.InventoryServiceHealth    Could not find Health Status EndPoint for Inventory Service
0 Kudos
Highlighted
Enthusiast
Enthusiast

This might help.

vAPI Endpoint out of memory issue in vSphere 6.x federated vCenter Server environments results in pa...

You may also try to increase JVM heap size of the vAPI Endpoint service. Let me know if you need help on that.

0 Kudos
Highlighted
Contributor
Contributor

We have the same issue with vSphere 6.5!! I reset the Service but the alarm is still there don't know what to do, please help! thanks

vSphere Web Client : Version 6.5.0 Build 5178943

VMware ESXi: Version 6.5.0 Build 4887370

0 Kudos
Highlighted
Contributor
Contributor

vAPI endpoint status changes to yellow in vCenter Server (2144715)

Regards Sandeep Kaushik
0 Kudos