Vmware View 6.1 Zero-Client disconnect

[projectserve]

We recently upgraded our VDI platform to VMWare View 6.1 using vSphere 6.0 and Nvidia vGPU (Grid K1). (HP T310 ZeroClient 4.7.1)

Some of our users complaining about disconnects, some users have this error more then others.

Mouse stops moving and after 30 seconds a disconnect.

Here are some log entries:

03/25/2015, 08:42:41.131> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: ==========>  New connection request <===========

03/25/2015, 08:42:41.131> LVL:2 RC:   0           AGENT :tera_agent_read_external_udp_address: RegQueryValueEx could not find the pcoip.external_udp_address value in the Software\Policies\Teradici\PCoIP\pcoip_admin_defaults key, sending no value to server.

03/25/2015, 08:42:41.131> LVL:2 RC:   0           AGENT :tera_agent_read_external_udp_address: RegQueryValueEx could not find the pcoip.external_udp_port value in the Software\Policies\Teradici\PCoIP\pcoip_admin_defaults key, sending no value to server.

03/25/2015, 08:42:41.131> LVL:2 RC:   0           AGENT :Client address is 0.0.0.0:0 (host order)

03/25/2015, 08:42:41.132> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: For Soft Host: Using Version 1 Tag

03/25/2015, 08:42:41.132> LVL:2 RC:   0             PRI :pcoip_agent_connect_req: {s_tag:0x5bf01e9d8f889fe5} Session ID for Soft Host: Tag:'W/AenY+In+UA' Value:5bf01e9d8f889fe5

03/25/2015, 08:42:41.132> LVL:2 RC:   0           AGENT :server_listen_on_addr is 0.0.0.0:0 (host order)

03/25/2015, 08:42:41.132> LVL:1 RC:   0           AGENT :pcoip_agent_connect_req:  type = 2

03/25/2015, 08:42:41.132> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: {s_tag:0x5bf01e9d8f889fe5} Session ID = 4; codec = 2.

03/25/2015, 08:42:41.132> LVL:1 RC:   0           AGENT :pcoip_agent_connect_req: debug trace log 1

03/25/2015, 08:42:41.133> LVL:2 RC:   0           AGENT :tera_agent_launch_server: {s_tag:0x5bf01e9d8f889fe5} Launching pcoip_server_win32

03/25/2015, 08:42:41.133> LVL:2 RC:   0           AGENT :tera_agent_launch_server: {s_tag:0x5bf01e9d8f889fe5} Optional log file path specified as "C:\ProgramData\VMware\VDM\logs\"

03/25/2015, 08:42:41.133> LVL:2 RC:   0           AGENT :tera_agent_launch_server: {s_tag:0x5bf01e9d8f889fe5} use_vmware_launcher = true.

03/25/2015, 08:42:41.135> LVL:2 RC:   0           AGENT :create_session_process: Opened pcoip server process handle 00000000000003DC

03/25/2015, 08:42:41.135> LVL:2 RC:   0           AGENT :tera_agent_launch_server: {s_tag:0x5bf01e9d8f889fe5}  Using VMWare's launcher code: worked [system is vista or newer, windows session id: 0x1, pid: 0x1e4, proc handle: 0x3dc].

03/25/2015, 08:42:41.381> LVL:2 RC:   0           AGENT :sSERVER_SESSION::agent_receiver_callback: message from A:srvr4;B:srvr0004 to A:srvr4, message = 00 00 00 00, len=144

03/25/2015, 08:42:42.135> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: {s_tag:0x5bf01e9d8f889fe5} [4] Waiting for ready message.

03/25/2015, 08:42:42.236> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: {s_tag:0x5bf01e9d8f889fe5} [4] Got ready message.

03/25/2015, 08:42:42.236> LVL:2 RC:   0           AGENT :selected server_addr is 0.0.0.0:4172 (host order)

03/25/2015, 08:42:42.236> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: {s_tag:0x5bf01e9d8f889fe5} [4] Connecting to server's mailbox.

03/25/2015, 08:42:42.236> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: {s_tag:0x5bf01e9d8f889fe5} [4] Sending session tag.

03/25/2015, 08:42:42.236> LVL:2 RC:   0           AGENT :sSERVER_SESSION::send_message: {s_tag:0x5bf01e9d8f889fe5} 136 bytes

03/25/2015, 08:42:42.236> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: {s_tag:0x5bf01e9d8f889fe5} [4] Sending session option #0 key='pcoip.enable_tera2800' value='1'.

03/25/2015, 08:42:42.236> LVL:2 RC:   0           AGENT :sSERVER_SESSION::send_message: {s_tag:0x5bf01e9d8f889fe5} 48 bytes

03/25/2015, 08:42:42.236> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: {s_tag:0x5bf01e9d8f889fe5} [4] Sending session option #1 key='pcoip.priority_level' value='4'.

03/25/2015, 08:42:42.236> LVL:2 RC:   0           AGENT :sSERVER_SESSION::send_message: {s_tag:0x5bf01e9d8f889fe5} 47 bytes

03/25/2015, 08:42:42.236> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: {s_tag:0x5bf01e9d8f889fe5} [4] Sending session option #2 key='pcoip.server_fqdn' value='PS-VDI-01.PROJECTSERVE.local'.

03/25/2015, 08:42:42.236> LVL:2 RC:   0           AGENT :sSERVER_SESSION::send_message: {s_tag:0x5bf01e9d8f889fe5} 71 bytes

03/25/2015, 08:42:42.236> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: {s_tag:0x5bf01e9d8f889fe5} [4] End of options: sending null-option message

03/25/2015, 08:42:42.236> LVL:2 RC:   0           AGENT :sSERVER_SESSION::send_message: {s_tag:0x5bf01e9d8f889fe5} 26 bytes

03/25/2015, 08:42:42.362> LVL:2 RC:   0           AGENT :sSERVER_SESSION::agent_receiver_callback: message from A:srvr4;B:srvr0004 to A:srvr4, message = 05 00 00 00, len=136

03/25/2015, 08:42:42.388> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: {s_tag:0x5bf01e9d8f889fe5} [4] Adding session to list.

03/25/2015, 08:42:42.388> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: {s_tag:0x5bf01e9d8f889fe5} [4] Total number of active sessions = 1

03/25/2015, 08:42:42.388> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: {s_tag:0x5bf01e9d8f889fe5} [4] Sending connection response ok.

03/25/2015, 08:42:42.388> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: {s_tag:0x5bf01e9d8f889fe5} [4] connection_response (end), 0

03/25/2015, 08:42:42.580> LVL:2 RC:   0           AGENT :sSERVER_SESSION::agent_receiver_callback: message from A:srvr4;B:srvr0004 to A:srvr4, message = 0b 00 00 00, len=16

03/25/2015, 08:42:42.602> LVL:2 RC:   0           AGENT :monitor_soft_hosts: {s_tag:0x5bf01e9d8f889fe5} MBX_CON_COMPLETE

03/25/2015, 08:42:42.602> LVL:2 RC:   0           AGENT :monitor_soft_hosts: {s_tag:0x5bf01e9d8f889fe5} connection_complete: SOFT - MBX_CON_COMPLETE

03/25/2015, 08:42:43.645> LVL:2 RC:   0           AGENT :monitor_soft_hosts: {s_tag:0x5bf01e9d8f889fe5} monitor thread: connection_complete(MBX_CON_COMPLETE) callback is successful

03/25/2015, 08:42:43.646> LVL:2 RC:   0           AGENT :mb_send_acknowledgement: {s_tag:0x5bf01e9d8f889fe5} Sending ack.

03/25/2015, 08:42:43.646> LVL:2 RC:   0           AGENT :sSERVER_SESSION::send_message: {s_tag:0x5bf01e9d8f889fe5} 8 bytes

03/25/2015, 08:42:43.748> LVL:2 RC:   0           AGENT :mb_send_acknowledgement: {s_tag:0x5bf01e9d8f889fe5} Succeeded to srvr0004

03/25/2015, 08:52:46.565> LVL:2 RC:   0           AGENT :monitor_soft_hosts: {s_tag:0x5bf01e9d8f889fe5} Server died.

03/25/2015, 08:52:46.666> LVL:2 RC:   0           AGENT :tera_agent_disconnect [soft host]: agent close code: 6, disconnect reason: 0

03/25/2015, 08:52:46.666> LVL:2 RC:   0           AGENT :tera_agent_disconnect: {s_tag:0x5bf01e9d8f889fe5} disconnect is ** NOT ** pending (hndl: 4, pid: 484, process handle: 000003dc)

03/25/2015, 08:52:46.666> LVL:2 RC:   0           AGENT :tera_agent_disconnect: {s_tag:0x5bf01e9d8f889fe5} Server process already exited (hndl: 4, pid: 484, process handle: 000003dc)

03/25/2015, 08:52:46.666> LVL:2 RC:   0           AGENT :tera_agent_finish_disconnect_thread: connection_closed 6

03/25/2015, 08:52:46.939> LVL:2 RC:   0           AGENT :sSERVER_SESSION::~sSERVER_SESSION: {s_tag:0x5bf01e9d8f889fe5} Closing pcoip server process handle 00000000000003DC

03/25/2015, 08:53:01.517> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: ==========>  New connection request <===========

03/25/2015, 08:53:01.518> LVL:2 RC:   0           AGENT :tera_agent_read_external_udp_address: RegQueryValueEx could not find the pcoip.external_udp_address value in the Software\Policies\Teradici\PCoIP\pcoip_admin_defaults key, sending no value to server.

03/25/2015, 08:53:01.518> LVL:2 RC:   0           AGENT :tera_agent_read_external_udp_address: RegQueryValueEx could not find the pcoip.external_udp_port value in the Software\Policies\Teradici\PCoIP\pcoip_admin_defaults key, sending no value to server.

03/25/2015, 08:53:01.518> LVL:2 RC:   0           AGENT :Client address is 0.0.0.0:0 (host order)

03/25/2015, 08:53:01.518> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: For Soft Host: Using Version 1 Tag

03/25/2015, 08:53:01.518> LVL:2 RC:   0             PRI :pcoip_agent_connect_req: {s_tag:0x3c5d2733bc4b5aa} Session ID for Soft Host: Tag:'A8XSczvEtaoA' Value:03c5d2733bc4b5aa

03/25/2015, 08:53:01.518> LVL:2 RC:   0           AGENT :server_listen_on_addr is 0.0.0.0:0 (host order)

03/25/2015, 08:53:01.518> LVL:1 RC:   0           AGENT :pcoip_agent_connect_req:  type = 2

03/25/2015, 08:53:01.518> LVL:2 RC:   0           AGENT :pcoip_agent_connect_req: {s_tag:0x3c5d2733bc4b5aa} Session ID = 5; codec = 2.

03/25/2015, 08:53:01.518> LVL:1 RC:   0           AGENT :pcoip_agent_connect_req: debug trace log 1

[Ray_handels]

We tested the new driver and although the amount of disconnects have decreased quite a bit it still wasn't the solution.

We did receive an email from VMWare that NVidia was able to reproduce the issue so I really really hope the next driver will have a fix. Seeing the last update took about 2 months my guess would be we will be seeing a correct driver somewhere in July.

[whibr]

I am also finding this disconnection issue happening in our environment.  So far, it seems to just be a random event among many of our Dell Wyse zero client users (dual-display, P25, 4.8 FW) and vGPU linked clone desktops pools.  I would like to try upgrading with the newer NVIDIA drivers, and see if that helps any as well.  Once I do, is there any way that you have found to reproduce this error?  Also, we have upgraded our K2 card gpu VBIOS in our older (circa 2013) GRID K2 card's to a newer version (Dell link).  I was wondering if anyone has done this VBIOS upgrade also?

[00Joerg00]

After several days of testing we found that working with 1vCPU helps! It is a Workaround until we get the NVIDA update.

[Ray_handels]

Not quite sure if it is already publicly available but we received an upgraded Windows driver (so just Windows) version 348.20.

We received it from VMWare themselves (we do have an open SR though) and i'm testing it right now.

Because we still have a multitude of disconnects my guess would be that I have a good feeling if this fixes our issues within a few days. Will keep you guys informed.

[elgwhoppo]

How's it going Ray? One of my peers updated to 348.07 2015.5.18 and that seems to have gotten rid of the PCoIP disconnects for him with 2 vCPU multimonitor setups.

[whibr]

Elgwhoppo,

We have upgraded to the 348.07 windows driver and the 346.68 vib a week ago, but that hasn't fixed our dual-display, 2vcpu session disconnects.  I also have a SR open and hope to get some feedback soon.  Our workaround plan is to start giving users a 24-in monitor instead of the two 19-in they are currently using.

[Ray_handels]

I must say i'm very happy with how the new driver turns out to be.

We are testing with about 20 people and normally a user had up to 2 disconnects every day. WIth the new driver we didn't have any disconnect at all for the last 4 days, so my guess is the new driver fixes it.

Be aware though that this is the 348.20 Windows driver NOT the 348.07 driver you can download at the NVidia site, we still had the issue with that driver. This driver is not yet publicly available because when installing i noticed it wasn't even signed yet so it seems to be extremely new.

The only thing you need to update is the windows driver. You do need to have the latest ESX driver though (the 346.68) which is downloadable at the NVidia site. These two drivers work together like a charm.

I'm not quite sure if i can actually realease the driver to you guys. I would suggest raising a call with VMWare and ask them for the 348.20 windows driver. My guess would be that the more people test the driver, the better they can eventually make it.

So one happy camper here

[projectserve]

So far no disconnects with the systems whom has the pre-release driver 348.20

[00Joerg00]

Hi Ray,

thanks for your information!

Do you have a download link for me? We are here 32 people who have over around disconnection every day every user.

[00Joerg00]

here me email address: joerg.schwettmann@pluemat.de

[whibr]

We have also began testing of the 348.20 beta driver from NVIDIA.  So far, after one day...zero random session disconnections from anyone that has experienced those before.  I will update if anything interesting develops.  I would suggest anyone wanting to test this windows display driver create a VMware SR, explain your environment and problem symptoms, and you should receive this driver to begin testing.

[00Joerg00]

Ok Ray, thanks I do this now.

[Ray_handels]

As said.

I'm not quite sure if i can release the driver. If you can't get it from VMWare, just let me know (PM if possible) and i will get it to you.

The problem is most defintly fixed with this driver. We didn't have any disconnects in the last few days.

I guess we will be seeing an official release somewhere in July? But at least we can proceed now with our vGPU enviornment..

[Jonathan_Filipp]

Hello!

Not quite sure how to Private message, but i have been trialing vgpu for the last month and these disconnections are killing progression..

Are you able to help me out and point me to the location of this magical fix ?

July is too long to wait to start testing our environment and i want to get this up and running asap!!

I do have active support subscription to all our environments and if it is easier to get it from vmware, i will pursue this tomorrow.

Thanks guys for the good information - i thought there was some weird environmental issue plaguing me until i bothered to google this today...

[Ray_handels]

I believe you need to have enough points to PM. But my guess is just opening a call with VMWare and refer to the new driver available for NVIdia.

I think if you refer to this KB article you should be able to get it quite quickly. If not possible if you provide me with a place put it i will give it a try..

[Jonathan_Filipp]

Thanks Ray,

I raised a ticket request pointing to this post. Hopefully will have a resolution soon!

Kind regards,

Jonathan

[00Joerg00]

With the beta driver we don't have disconnections anymore! Thanks Ray!

[Ray_handels]

Good to hear that the issue is resolved. We do still see some VDI machines "hanging" for a few seconds and then move on with what they are doing. It seems as if they have fixed the disconnects but the driver still seems to do some funcky stuff from time to time. But this is a beta driver so hopefully the GA driver is better.

I heard from my VMWare contact that ETA was End June so it was due already. My guess it will be this or next week that we will see the driver GA.

If it is there i will post (or maybe if someone is quicker than me please let us know )

[Ray_handels · Accepted solution]

New Windows driver is GA and the ESX driver stayed the same so luckily only an update of the Windows driver.

Link is here.

NVIDIA DRIVERS NVIDIA GRID VGPU SOFTWARE RELEASE 346.68/348.27 WHQL

[iforbes]

I'm also running vGPU for a bunch of desktops. My issue is opposite. The vGPU desktops operate just fine. I'm going to update to the latest nVidia drivers just in case though. My issue is that the non-vGPU desktops are not allowing me to log in. What happens is that I get a black screen during login and disconnect a few seconds later. When I take a look at the View desktop pcoip server logs I see the exact same errors you've posted. I get the same disconnect code 105. Normally I would think this was a PCoIP misconfiguration for external users coming through the firewall, but I'm having the same symptoms for internal users that don't go through a firewall. Any ideas?