9 Replies Latest reply on Nov 12, 2009 8:01 PM by DHassel

    Management via web/viclient failing in 2.0.1 + 2.0.2

    mcooper7290 Lurker

       

      I was recently forced to update a Red Hat Enterprise Linux 5.3 system to have all current updates (yum update).  After performing the system update the VMware Server web management page keeps failing.   I can get logged in but when I client on most any vm guest and try most any operation (like Play/Resume) the management page displays "Loading..." in all th enormal places but never resumes working.  I have to run "/etc/init.d/vmware-mgmt restart" to get it to respond again.  Of course, it only works enough for me to log back in and try one operation.

       

       

      The guests set to autostart do run and get started.

       

       

      This problem occurs for multiple guests.  It occurs with VMware Server 2.0.1.  I upgraded to 2.0.2 and have the same problem.

       

       

      I installed viclient 2.5 from vmware server 2.0.0 and it behaves in a similar manner.  After most any type of operation on a vm guest viclient stops working stating it's lost contact with the server.  At the same time the web mgmt page stops working as well.  A "vmware-mgmt restart" gets viclient going again, but it fails once I perform an operation.

       

       

      Any suggestions?

       

       

      mike

       

       

        • 1. Re: Management via web/viclient failing in 2.0.1 + 2.0.2
          guyrleech Virtuoso

           

           

           

           

          Not seen this myself but ...

           

           

           

          Does vmware-config.pl complete without error?

           

           

          Anything suspicious looking in the logs either in /var/log/vmware or for the log for a VM you try and start (which will be in the same folder as the .vmx)?

           

           

           

          Guy Leech

          VMware vExpert 2009

           

          ---

          If you found this or any other answer useful please consider the use of the Helpful or Correct buttons to award points.

           

           

          • 2. Re: Management via web/viclient failing in 2.0.1 + 2.0.2
            rmuetze Lurker

             

            Hi experts,

             

             

             

             

             

            same here. I'm totally lost for a couple of days now after updating to server 2.0.2 and running "yum update" just before that. So I'm not sure if this is vmware or os related. But back to the facts. I experience a behaviour like this:

             

             

            - can't connect to the web VI, Trying this will yield a blank page and an error " The connection to the server was reset while the page was loading." when trying to refresh the page (a task that proved being helpful in the past)

             

             

            - vmware-mgmt restart will restore the behaviour for another try: blank page and error after reload

             

             

            - vmrun won't connect to the server:

             

             

            There are some logfiles available:

             

             

              # ll webAccess/

            total 8

            -rw-rr 1 root root    0 Nov  5 11:03 client.log

            -rw-rr 1 root root    0 Nov  4 16:27 objectMonitor.log

            -rw-rr 1 root root 3888 Nov  5 11:20 proxy.log

            -rw-rr 1 root root    0 Nov  5 11:03 timer.log

            -rw-rr 1 root root    0 Nov  5 11:03 unitTest.log

            -rw-rr 1 root root    0 Nov  4 16:27 updateThread.log

            -rw-rr 1 root root    0 Nov  4 16:27 viewhelper.log

             

             

            Any idea why most of the webAccess logs are  empty?However, proxy log shows:

             

             

            queryTargetOption failed: (0)null

            Updates thread error notification. PrevError=null, NewError=UnknownRemoteException, NewErrorMessage=java.net.SocketException: Connection reset

            Error processing action request /view/fetch : java.net.SocketException: Connection reset

            Updates thread error notification. PrevError=null, NewError=UnknownRemoteException, NewErrorMessage=(0)null

             

             

             

            Right now, it looks like I found some answers. From reading the logs I found a lot of "connection refused" and "connection reset" errors in WebAccess' log files. I digged somewhat deeper and finally found this in hostd.log:

             

             

            SSL Handshake on client connection failed: SSL Exception:

            SSL Handshake on client connection failed: SSL Exception:

            RecordOp ASSIGN: guest.disk, 256

            PendingRequest: HTTP Transaction failed, closing connection. Reason: Connection reset by peer, class type is: N7Vmacore15SystemExceptionE

             

             

            and

             

             

            SSL Handshake on client connection failed: SSL Exception: error:140D9115:SSL routines:SSL_GET_PREV_SESSION:session id context uninitialized

            SSL Handshake on client connection failed: SSL Exception: error:140D9115:SSL routines:SSL_GET_PREV_SESSION:session id context uninitialized[2009-11-05 11:39:38.878 'Proxysvc Req00198' 1081432384 warning] PendingServerStrm: client write-completion error: SSL Exception: error:00000001:lib(0):func(0):reason(1)

            PendingServerStrm: client write-completion error: SSL Exception: error:00000001:lib(0):func(0):reason(1)

            RecordOp ASSIGN: guest.disk, 16

            PendingServerStrm: client write-completion error: SSL Exception: error:00000001:lib(0):func(0):reason(1)

            PendingServerStrm: client write-completion error: SSL Exception: error:00000001:lib(0):func(0):reason(1)

            PendingServerStrm: client write-completion error: SSL Exception: error:00000001:lib(0):func(0):reason(1)

            PendingServerStrm: client write-completion error: SSL Exception: error:00000001:lib(0):func(0):reason(1)

             

             

            All the problems look SSL related. I was able to connect after deleting the certificate exception from firefox. However, the problems reoccur from time to time.

             

             

             

             

             

            Ah, and yes, vmware-config.pl completed without errors (several times ;o)

             

             

             

             

             

             

             

             

             

             

             

            • 3. Re: Management via web/viclient failing in 2.0.1 + 2.0.2
              postlethwaite Lurker

               

              I am also running RHEL 5.

               

               

              The web interface will start, but as soon as I make changes to a VM, The interface is gone and dead.

               

               

              I am running RHEL 5.4 64 bit with CentOS 5.4 64bit.

               

               

               

              I have restarted, reinstalled VMware Server and the CentOS guest. The guest remain alive while the VMware Tomcat server is dead.

               

               

               

              I am running SELinux, this does not appear to be the problem. The problem exists while SELinux is disabled.

               

               

               

              It does not matter if guest are running or not, the web app dumps and

              cannot be accessed. Any attempts to refresh the URL does not revive the

              server.

               

               

               

              02/Nov/2009:19:13:02.935ERRORcom.vmware.vim.VimChannel: HTTP request errored (12029 Unknown).

               

              02/Nov/2009:19:13:03.109ERRORcom.vmware.vim.VimChannel: HTTP request errored (12029 Unknown).

               

              02/Nov/2009:19:13:03.113ERRORvmtn.core.RuntimeException:

              Error or Timeout or Fault.at

              com.vmware.vim.inventory.InventoryViewManagedObject.__onInventoryViewRetrieved([vmtn.app.data.Respons...)at

              vmtn.app.data.Request.__fireResponseEvent(25,

              [vmtn.net.HttpResponse..., null)at

              vmtn.app.data.Request._fireResponseErrored([vmtn.net.HttpResponse...)at

              vmtn.app.data.AbstractChannel.__handleResponseError(vmtn.app.data.Request@...,

              [vmtn.net.HttpResponse...)at

              vmtn.app.data.AbstractChannel.__onHttpResponse([vmtn.net.HttpResponse...)at

              vmtn.util.event.ListenerList.fire([vmtn.net.HttpResponse...)at

              vmtn.net.AsyncHttpRequest.__onStateChange()

               

               

               

              I am accessing the site with the debug window open:

               

               

               

              https://my-domain:8333/ui/debug.html#

               

               

               

              After this crash the web app is dead and no longer logs to any of the logs owned byVMware.

               

               

               

              The server and guests remain accessible by the CLI. I am running headless machines (No X or Windows Manager).

               

               

               

              This problem exists in 2.01 and 2.02.

               

               

              I currently run a Windows Server 2008 machine with VMware Server 2.01 which is stable.

               

               

               

              I am running on Dell servers. I am not sure, but I noticed VMs that fail are the ones that have a DVD drive enabled.

               

               

              Jeremy

               

               

               

               

               

              • 4. Re: Management via web/viclient failing in 2.0.1 + 2.0.2
                Viveo Lurker

                Hi,

                 

                I have exactly the same problem. I was running vmware server 2.0.1 x86_64 on RedHat Enterprise 5 (64 bit). After I upgraded RedHat 5 to the latest release, the vmware web console started crashing all the time. I cannot manage the vmware guests anymore. I upgraded vmware server to 2.0.2. and ran vmware-config.pl without any errors. But the web management continues to fail persistently. Any help would be much appreciated. 

                 

                Below is a extract from proxy.log

                 

                 

                Error in the method call create : java.net.ConnectException: Connection refused

                Error processing action request /view/treenav : java.net.ConnectExcep

                tion: Connection refused

                Dependent property VM.runtime not loaded by toolkit.

                Updates thread error notification. PrevError=null, NewError=UnknownRemoteException, NewErrorMe

                ssage=(0)null

                Updates thread error notification. PrevError=null, NewError=UnknownRemoteException, NewErrorMe

                ssage=(0)null

                queryConfigOption failed: (0)null

                queryConfigOption failed: java.net.ConnectException: Connection refused

                Updates thread error notification. PrevError=null, NewError=UnknownRemoteException, NewErrorMe

                ssage=(0)null

                Updates thread error notification. PrevError=null, NewError=UnknownRemoteException, NewErrorMe

                ssage=(0)null

                Updates thread error notification. PrevError=null, NewError=UnknownRemoteException, NewErrorMe

                ssage=(0)null

                [2009-11-10 16:49:17,472,ContainerBackgroundProcessor[StandardEngine[Catalina]]<=>,TreeNavPropertySet] Error cleaning up TreeNavPropertySet

                com.vmware.core.UnknownRemoteException: java.net.ConnectException: Connection refused

                at com.vmware.vim.managedobject.View.destroyView(View.java:31)

                at com.vmware.vim.adapter.builder.TreeNavPropertySet.destroy(Unknown Source)

                at com.vmware.vim.adapter.builder.TreeNavPropertySet.onRemove(Unknown Source)

                at com.vmware.vim.adapter.builder.PropertySetManagerImpl.rmvPropertySet(Unknown Source)

                at com.vmware.vim.adapter.builder.PropertySetManagerImpl.rmvAllPropertySets(Unknown Source)

                at com.vmware.vim.adapter.builder.PropertySetManagerImpl.discard(Unknown Source)

                at com.vmware.vim.adapter.builder.PropertySetContextImpl.destroy(Unknown Source)

                at com.vmware.vim.adapter.RequestProcessor.destroy(Unknown Source)

                at com.vmware.vim.adapter.broker.ServerBrokerImpl.cleanup(Unknown Source)

                at com.vmware.webaccess.listener.SessionListener.sessionDestroyed(Unknown Source)

                at org.apache.catalina.session.StandardSession.expire(StandardSession.java:702)

                at org.apache.catalina.session.StandardSession.isValid(StandardSession.java:592)

                at org.apache.catalina.session.ManagerBase.processExpires(ManagerBase.java:682)

                at org.apache.catalina.session.ManagerBase.backgroundProcess(ManagerBase.java:667)

                at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1316)

                at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1601)

                at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1610)

                at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1610)

                at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1590)

                at java.lang.Thread.run(Thread.java:595)

                Caused by: java.net.ConnectException: Connection refused

                at org.apache.axis.AxisFault.makeFault(AxisFault.java:101)

                at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:154)

                at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)

                at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)

                at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)

                at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)

                at org.apache.axis.client.Call.invokeEngine(Call.java:2784)

                at org.apache.axis.client.Call.invoke(Call.java:2767)

                at org.apache.axis.client.Call.invoke(Call.java:2443)

                at org.apache.axis.client.Call.invoke(Call.java:2366)

                at org.apache.axis.client.Call.invoke(Call.java:1812)

                at com.vmware.vim.VimBindingStub.destroyView(VimBindingStub.java:31784)

                at com.vmware.vim.managedobject.View.destroyView(View.java:27)

                ... 19 more

                Caused by: java.net.ConnectException: Connection refused

                at java.net.PlainSocketImpl.socketConnect(Native Method)

                at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)

                at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)

                at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)

                at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)

                at java.net.Socket.connect(Socket.java:520)

                at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source)

                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

                at java.lang.reflect.Method.invoke(Method.java:585)

                at org.apache.axis.components.net.DefaultSocketFactory.create(DefaultSocketFactory.java:153)

                at org.apache.axis.components.net.DefaultSocketFactory.create(DefaultSocketFactory.java:120)

                at org.apache.axis.transport.http.HTTPSender.getSocket(HTTPSender.java:191)

                at org.apache.axis.transport.http.HTTPSender.writeToSocket(HTTPSender.java:404)

                at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:138)

                ... 30 more

                [2009-11-10 16:49:17,489,ContainerBackgroundProcessor[StandardEngine[Catalina]]<=>,ServiceConnection] java.net.ConnectException: Connection refused

                [2009-11-10 16:49:17,506,ContainerBackgroundProcessor[StandardEngine[Catalina]]<=>,TreeNavPropertySet] Error cleaning up TreeNavPropertySet

                com.vmware.core.UnknownRemoteException: java.net.ConnectException: Connection refused

                at com.vmware.vim.managedobject.View.destroyView(View.java:31)

                at com.vmware.vim.adapter.builder.TreeNavPropertySet.destroy(Unknown Source)

                at com.vmware.vim.adapter.builder.TreeNavPropertySet.onRemove(Unknown Source)

                at com.vmware.vim.adapter.builder.PropertySetManagerImpl.rmvPropertySet(Unknown Source)

                at com.vmware.vim.adapter.builder.PropertySetManagerImpl.rmvAllPropertySets(Unknown Source)

                at com.vmware.vim.adapter.builder.PropertySetManagerImpl.discard(Unknown Source)

                at com.vmware.vim.adapter.builder.PropertySetContextImpl.destroy(Unknown Source)

                at com.vmware.vim.adapter.RequestProcessor.destroy(Unknown Source)

                at com.vmware.vim.adapter.broker.ServerBrokerImpl.cleanup(Unknown Source)

                at com.vmware.webaccess.listener.SessionListener.sessionDestroyed(Unknown Source)

                at org.apache.catalina.session.StandardSession.expire(StandardSession.java:702)

                at org.apache.catalina.session.StandardSession.isValid(StandardSession.java:592)

                at org.apache.catalina.session.ManagerBase.processExpires(ManagerBase.java:682)

                at org.apache.catalina.session.ManagerBase.backgroundProcess(ManagerBase.java:667)

                at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1316)

                at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1601)

                at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1610)

                at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1610)

                at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1590)

                at java.lang.Thread.run(Thread.java:595)

                Caused by: java.net.ConnectException: Connection refused

                at org.apache.axis.AxisFault.makeFault(AxisFault.java:101)

                at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:154)

                at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)

                at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)

                at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)

                at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)

                at org.apache.axis.client.Call.invokeEngine(Call.java:2784)

                at org.apache.axis.client.Call.invoke(Call.java:2767)

                at org.apache.axis.client.Call.invoke(Call.java:2443)

                at org.apache.axis.client.Call.invoke(Call.java:2366)

                at org.apache.axis.client.Call.invoke(Call.java:1812)

                at com.vmware.vim.VimBindingStub.destroyView(VimBindingStub.java:31784)

                at com.vmware.vim.managedobject.View.destroyView(View.java:27)

                ... 19 more

                Caused by: java.net.ConnectException: Connection refused

                at java.net.PlainSocketImpl.socketConnect(Native Method)

                at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)

                at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)

                at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)

                at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)

                at java.net.Socket.connect(Socket.java:520)

                at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source)

                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

                at java.lang.reflect.Method.invoke(Method.java:585)

                at org.apache.axis.components.net.DefaultSocketFactory.create(DefaultSocketFactory.java:153)

                at org.apache.axis.components.net.DefaultSocketFactory.create(DefaultSocketFactory.java:120)

                at org.apache.axis.transport.http.HTTPSender.getSocket(HTTPSender.java:191)

                at org.apache.axis.transport.http.HTTPSender.writeToSocket(HTTPSender.java:404)

                at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:138)

                ... 30 more

                [2009-11-10 16:49:17,521,ContainerBackgroundProcessor[StandardEngine[Catalina]]<=>,ServiceConnection] java.net.ConnectException: Connection refused

                Updates thread error notification. PrevError=null, NewError=UnknownRemoteException, NewErrorMe

                ssage=(0)null

                Updates thread error notification. PrevError=null, NewError=UnknownRemoteException, NewErrorMe

                ssage=(0)null

                 

                Pedro Monteiro

                • 5. Re: Management via web/viclient failing in 2.0.1 + 2.0.2
                  xkey Novice

                  I am running into same problems on CentOS 5.4 with latest kernel and "yum update" right before upgraded to latest 2.0.2.

                   

                  I was going to go back to old kernel version with 2.0.1, but couldn't find the old version of vmware server.

                   

                  btw, is vmwre server dying coz vmware ESXi is replacing as the mainstream free version of virtual solution for community users? if so, i'd like to move away from vmware server, although migration is a pain of ass

                  • 6. Re: Management via web/viclient failing in 2.0.1 + 2.0.2
                    binro Lurker

                    This also occurs with the latest Karmic release of Ubuntu (9.10 32 bit, desktop and server are my test machines) and did not occur with 8.04 which was installed prior to Karmic. The web UI for 2.0.1 or 2.0.2 dies - in a seemingly random manner and needs frequent restarts from the init.d mgmt script. The core vmx server processes are  fine and the guests simply keep on ticking but it does make management of them a little tricky shall we say.

                     

                     

                    I applied the patch to get the VMware server 2 install working as described on the help.ubuntu.com forums but based on this Thread history this is clearly not a Ubuntu unique issue.

                     

                     

                     

                     

                     

                     

                     

                     

                     

                    robin

                    • 7. Re: Management via web/viclient failing in 2.0.1 + 2.0.2
                      Dakaix Lurker

                      I am also seeing this issue on CentOS 5.4 x86_64 running the 2.6.18-164.6.1.el5 kernel and VMware Server 2.0.2. Restarting the management service does bring it up, but for me it will crash sometimes upon the subsequent login!

                       

                      Cheers,

                       

                      Dakaix.

                      • 8. Re: Management via web/viclient failing in 2.0.1 + 2.0.2
                        vojtek Lurker

                         

                        Hi,

                         

                         

                        I also experience trouble with running VMware Server 2.0.1 and 2.0.2 on RHEL 5.4. 

                         

                         

                        I found that the problem is related to VMware Server 2.0.1 and 2.0.2 were not build on new libraries (mainly C/C++) and kernel delivered by RHEL 5.4. Due to that, vmware-hostd process is crashing. I found information on other forum VMware Server is build on RHEL5.1.

                         

                         

                        Modules that are loaded into the system are compiled for kernel 2.6.18-8 which is even more than 20 general kernel updates ago from current RHEL version which is using 2.6.18-164.6.1. I have to keep the OS updated due to security bugs and issues.

                         

                         

                         

                         

                         

                        Why VMware Server software is still build on RHEL 5.1 release and does not take into account any updates of the hosting OS.

                         

                         

                        Can anyone answer that problem?

                         

                         

                         

                         

                         

                        vojtek

                         

                         

                        • 9. Re: Management via web/viclient failing in 2.0.1 + 2.0.2
                          DHassel Lurker

                           

                          I have had the same issue & my best solution was to re-install Red Hat 5.3 & VMware then worked fine.  Just DO NOT UPDATE to 5.4 via RHN.  Hopefull VMWare will come out w/ an update or patch soon to to address this issue.

                           

                           

                           

                           

                           

                          Dave Hassel