VMware {code} Community
alpsr
Contributor
Contributor

Getting java.lang.OutOfMemoryError: PermGen space error on plugin deployment

Hi,

Once in a while we are getting out of memory exception while deploying our plugin. Has anyone else seen such issue?

[2015-12-12T02:56:24.471Z] [WARN ] http-bio-9090-exec-62        70010579 100060 200009 org.springframework.flex.core.DefaultExceptionLogger              The following exception occurred during request processing by the BlazeDS MessageBroker and will be serialized back to the client:  flex.messaging.MessageException: PermGen space

    at flex.messaging.services.RemotingService.serviceMessage(RemotingService.java:225)

    at flex.messaging.MessageBroker.routeMessageToService(MessageBroker.java:1400)

    at flex.messaging.endpoints.AbstractEndpoint.serviceMessage(AbstractEndpoint.java:1011)

    at flex.messaging.endpoints.AbstractEndpoint$$FastClassByCGLIB$$1a3ef066.invoke(<generated>)

    at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)

    at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689)

    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)

    at org.springframework.flex.core.MessageInterceptionAdvice.invoke(MessageInterceptionAdvice.java:66)

    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)

    at org.springframework.aop.framework.adapter.ThrowsAdviceInterceptor.invoke(ThrowsAdviceInterceptor.java:124)

    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)

    at org.springframework.aop.framework.Cglib2AopProxy$FixedChainStaticTargetInterceptor.intercept(Cglib2AopProxy.java:573)

    at com.vmware.vise.messaging.endpoints.AMFEndpoint$$EnhancerByCGLIB$$5ef4832b.serviceMessage(<generated>)

    at flex.messaging.endpoints.amf.MessageBrokerFilter.invoke(MessageBrokerFilter.java:103)

    at flex.messaging.endpoints.amf.LegacyFilter.invoke(LegacyFilter.java:158)

    at flex.messaging.endpoints.amf.SessionFilter.invoke(SessionFilter.java:44)

    at flex.messaging.endpoints.amf.BatchProcessFilter.invoke(BatchProcessFilter.java:67)

    at flex.messaging.endpoints.amf.SerializationFilter.invoke(SerializationFilter.java:166)

    at flex.messaging.endpoints.BaseHTTPEndpoint.service(BaseHTTPEndpoint.java:291)

    at com.vmware.vise.messaging.endpoints.AMFEndpoint$$EnhancerByCGLIB$$5ef4832b.service(<generated>)

    at org.springframework.flex.servlet.MessageBrokerHandlerAdapter.handle(MessageBrokerHandlerAdapter.java:109)

    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)

    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)

    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)

    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:789)

    at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)

    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)

    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)

    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)

    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)

    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)

    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)

    at com.vmware.vise.security.SessionManagementFilter.doFilterInternal(SessionManagementFilter.java:82)

    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)

    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)

    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)

    at com.vmware.vsphere.client.logging.MDCLogFilter.doFilterInternal(MDCLogFilter.java:41)

    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)

    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)

    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)

    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)

    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)

    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)

    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)

    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)

    at org.eclipse.virgo.web.tomcat.support.ApplicationNameTrackingValve.invoke(ApplicationNameTrackingValve.java:33)

    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)

    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)

    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)

    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)

    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)

    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:315)

    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

    at java.lang.Thread.run(Unknown Source)

java.lang.OutOfMemoryError: PermGen space


laurentsd‌ : Do you know what could be going wrong or if you have any work around for this?

Reply
0 Kudos
7 Replies
laurentsd
VMware Employee
VMware Employee

I assume this is happening on your dev environment.

Did you see the note in SDK-Setup.html to increate your JVM memory?

Try this or higher:

-XX:MaxPermSize=512m

Reply
0 Kudos
alpsr
Contributor
Contributor

This is not happening on my dev environment, it is happening on one of our vCenters in lab where we are testing our plugin deployment.

Reply
0 Kudos
laurentsd
VMware Employee
VMware Employee

You should be able to increase the JVM memory for Virgo in your vCenter setup.  See JAVA_OPTS in /usr/lib/vmware-vsphere-client/server/bin/dmk.sh

I think there are some admin doc or KB articles about configuring your VCSA for higher memory requirements but I don't remember where they are.

Reply
0 Kudos
virat1234
Enthusiast
Enthusiast

is there a way to change this in a windows vCenter?
Reply
0 Kudos
Raj1988
Enthusiast
Enthusiast

Hi ,

In vCenter 6.x Windows :-

C:\ProgramData\VMware\vCenterServer\cfg  and open the file service-layout.mfx in notepad and change the MaxPermMB .

Then restart the services to take effect .

Reply
0 Kudos
ryantodd
Contributor
Contributor

laurentsd, service-layout.mfx has a "Do not edit this file by hand." warning in it. Is there a way in windows VC 6.0 to up the maxpermMB for, say, the vsphere web client? We're running into issues with a customer who is running into out of permgen space errors, and we suspect that upping the maxpermMB for the web client will resolve the issue, but we are concerned about changing this file by hand.

Also, there is the note:

# When modifying ram sizes the OS line must be adjusted such that:

# sum(tinyMB) + sum(MaxPermMB) = 8000

# sum(largeMB) + sum(MaxPermMB) = 32000

So, does one really need to decrease the OS tinyMB and largeMB sizes by 100 when increasing the web client permgenMB by 100?

Also, the check for the settings should be:

$ awk '!/^#/{t+=$5;l+=$6;p+=$7;print}END{print t,l,p;print t+p,l+p}' service-layout.mfx

(the comments were specifying the wrong columns, 6,7,8)

Reply
0 Kudos
laurentsd
VMware Employee
VMware Employee

Ryan,

yes you can try increasing the MaxPermDB value for vspherewebclientsvc in service-layout.mfx on that windows setup.

You'll need to decrease one or more values in both tinyMB and largeMB columns to comply to the sum rules.

I don't believe the deltas needs to be in 100s.

What size is that environment?

Let me know how it goes.

--Laurent

Reply
0 Kudos