VDM broker fails to service external requests after several hours

[Dave_O]

I have set up VDI with a VDM broker for both internal & external use. All works fine apart from the fact that externally after a few hours (7 or 😎 the broker stops servicing vm client requests. Prior to that everything is fine. The broker event log show a few errors (listed below). I have checked the VDM services on the broker and they are running OK. A reboot of the VDM server (which is virtual 2Gb menory single CPU) resolves the issue, but I don't want to keep doing that if it can be avoided. I have load balanced with 2 identical servers to try and reduce the impact but this is having no effect since the relevant services are still running on VDM server and hence there is no trigger to fail over. I know there are solutions address this but they do not really address the real issue, just mitigate against it. Any help or guidance would be appreciated.

Logs

!https://10.229.131.29/admin/images/small_events.gif!ASSERT: id == null (more ...) !https://10.229.131.29/admin/images/error16x16.gif!ERROR	03/21/2008 03:28:39 PM	Assert	TP-Processor2
!https://10.229.131.29/admin/images/small_events.gif!ASSERT: id == null (more ...)	!https://10.229.131.29/admin/images/error16x16.gif!ERROR	03/21/2008 03:28:39 PM	Assert	TP-Processor2
!https://10.229.131.29/admin/images/small_events.gif!Error retireving user for SID null. (more ...)	!https://10.229.131.29/admin/images/error16x16.gif!ERROR	03/21/2008 03:28:39 PM	Util	TP-Processor2
(Request143) Request failed: com.vmware.vdi.ob.tunnelservice.bk: Failed whilst returning body: java.io.IOException: Broken pipe com.vmware.vdi.ob.tunnelservice.dq.b(SourceFile:164) com.vmware.vdi.ob.tunnelservice.bk: Failed whilst returning body: java.io.IOException: Broken pipe at com.vmware.vdi.ob.tunnelservice.l.a(SourceFile:388) at com.vmware.vdi.ob.tunnelservice.l.b(SourceFile:419) at com.vmware.vdi.ob.tunnelservice.l.a(SourceFile:226) at com.vmware.vdi.ob.tunnelservice.dq.b(SourceFile:162) at com.vmware.vdi.ob.tunnelservice.d.run(SourceFile:167) at java.lang.Thread.run(Unknown Source) Caused by: java.io.IOException: Broken pipe at simple.http.MonitoredOutputStream.destroy(Unknown Source) at simple.http.MonitoredOutputStream.write(Unknown Source) at simple.http.ResponseStream.write(Unknown Source) at com.vmware.vdi.ob.tunnelservice.dq.a(SourceFile:123) at com.vmware.vdi.ob.tunnelservice.l.a(SourceFile:386) ... 5 more !https://10.229.131.29/admin/images/small_events.gif!(Request143) Request failed: com.vmware.vdi.ob.tunnelservice.bk: Failed whilst returning body: java.io.IOException: Broken pipe com.vmware.vdi.ob.tunn (more ...)	!https://10.229.131.29/admin/images/error16x16.gif!ERROR	03/21/2008 03:27:58 PM	SimpleAJPService	AJP-43
!https://10.229.131.29/admin/images/small_events.gif!ASSERT: id == null (more ...)	!https://10.229.131.29/admin/images/error16x16.gif!ERROR	03/21/2008 03:23:22 PM	Assert	TP-Processor3
!https://10.229.131.29/admin/images/small_events.gif!ASSERT: id == null (more ...)	!https://10.229.131.29/admin/images/error16x16.gif!ERROR	03/21/2008 03:23:22 PM	Assert	TP-Processor3
!https://10.229.131.29/admin/images/small_events.gif!Error retireving user for SID null. (more ...)	!https://10.229.131.29/admin/images/error16x16.gif!ERROR	03/21/2008 03:23:22 PM

[dougdavis22]

Dave O,

We have had a similar issue on occasion with our VDM server - all of a sudden, it will stop serving up the web client pages & admin pages. After a period of 10 or so minutes, it seems to come back to life. Last week, we rebooted the VDM Connection server, and it's been fine since, but no doubt we'll hit a point soon when it happens again.

My theory is that it's something to do with idle sessions building up, but the resource usage of the server doesn't suggest any issues.

I'm monitoring it anyway, and will be letting VMware know if I continue to have issues.

Doug.

[Dave_O]

Is your VDM servers physical or virtual? I'm trying to decide if it is worth building up a physical machine to see if that has any effect. I know it's unlikely, but needs must when the devil drives.

[vheff]

Has anyone still experienced these issues where the VDM server stops responding to web requests with the main page and admin page? I'm having these issues 4 or 5 times a day and I've been restarting the VDM Connection Server service to get it working each time.

[Dave_O]

Yes.

Still having exactly the same problem. I have set up a scheduled reboot 4 times per day on each VDM server!! This is not a solution. I am beginning to lose patience with this. My terminal server just keeps working. I'd rather use VDI but if its this unreliable then....

[vheff]

I've logged this with VMware and I'm waiting for their response. We are trying to showcase VMware technologies at my company (government organisation) and so far it is struggling to impress with these issues. We've also got an issue where logged on users show as 'null' in the admin interface. Have you seen this?

[Dave_O]

I have just logged a support call as well. And yes they are showing up as null. I am aware that there are some issues with Java on the VDM server and that they are manifest as lost connections but this is usually associated with the service stopping. In my case the service does not stop.

[vheff]

My service does not stop either, but when I restart it then it comes back to life. Let me know if you hear anything, and I'll do likewise.

Ray

[kjohnston]

How many virtual machines do you have configured? Also - at the point where the VDM server stops servicing requests, can you see how much memory they are using?

Does this only affect external clients? Can clients internally still get desktops through the broker?

[vheff]

I'm using VDM with 7 x virtual desktops, and VDM is only used internally at the moment. When it stops servicing requests, it is important to note that the VDM Connection Server server is still running, it's still listening on port 443, but when I try and access the page I get a 'page not found' error. The memory usage on each virtual desktop is minimal, and this has even done this with nobody connected.

What is aparent, is every time it has happened, I have been using the admin interface (https://vdmserver/admin) to display the virtual machines tab. This may be coincidence?

[kjohnston]

Sorry - I mean to ask - what is the memory usage of the VDM connection server. There should be three processes called "ws_java_bridge". Can you check their memory usage?

So in your case it sounds like you still can access the admin UI?

[vheff]

No, I'm getting two seperate issues. Sometimes it won't display the web page (including the admin page), and when it is working the rest of the time, usernames always show as null. I don't know about the memory usage, but I will have a look tomorrow.

[TomHowarth]

Did you geta chance to look at the memory usage

Tom Howarth

VMware Communities User Moderator

[Frank_Taylor1]

The exception in your events output is unlikely to be the cause of the server not accepting connections. You get this exception when the client closes its connection for an HTTP that is still to be returned.

We have seen other instances of the service not accepting connections, but as yet have been unable to reproduce this problem in the lab. We see that the reset happens during the SSL handshake.

Would you be able to test a potential work around for this?

Thanks,

Frank.

[CHASYS]

I see the same errors in the log but agree that these are not the cause of the server not accepting connections. One of our clients has exactly the same issue with the VDM server webpages returning page not found although the services are still running. Restarting the VDM services does not fix the problem, but a full server reboot does result in the server accepting connections. This really is a problem that needs a solution ASAP as it makes VDI virtually unuseable.

[Frank_Taylor1]

I agree that this is an important issue, however we are unable to reproduce this problem. I am looking for sites with the problem to help us diagnose what is going on. Do you know anyone who can help?

Thanks,

Frank.

[CHASYS]

If you require any further logs or other information I shall be happy to see if we can help, as we are quite desperate to find a solution to this problem.

Regards

Christian

[vheff]

I've not been in the office for a few weeks, but I'm now back. The first issue is where the user shows as 'null' is in the attached screenshots, and the other problem where users get disconnected is still happening although only twice this week!

I can confirm that when users are disconnected, I can't access port 443 (telnet vdm_server 443), the service is running, and a restart does fix it. I have applied the following (http://technet2.microsoft.com/windowsserver/en/library/8d3a9f4d-13d1-4280-ac57-30242504d8ba1033.mspx) but not sure if that has fixed it yet.

[SergioB]

Are you using AD Groups or AD Users for the Pool entitle settings? We hade the same issue on are VDM by using AD Groups for the entitle of Users. After adding an E-Mail address to the Group the problems disappeared. And the username was displayed on VDM admin page.

Greetings

Sergio

[PerryM]

Hi. We're having the same problem. The user shows up as "null". I am using AD Groups for entitlements. I will try adding an email address to the AD Group which has been entitled. I imagine it will take awhile for this change to kick in. I wonder if it will work if I just entitle the users instead of using a group...?

Has VMware acknowledged this problem ?