VMWare View 4.6 PCoIP tunneling problem. UDP doesn't get tunneled

[heikkitoivanen]

Hi,

I've got the "classic black screen"-issue.
So when I try to login to a virtual desktop, I'm nicely auhtenticated, I can select a desktop pool, but once the desktop launches,
I just get a black screen, and afterawhile it times out.

I've read the the manuals, the document http://communities.vmware.com/docs/DOC-14974, written by Mark benson; Watched the video; Verified and re-verified the 3 magic steps;
Read Paul Slager's blog, http://paulslager.com/?p=1300 and still i'm stuck. I've read the (some of the) logs from the View Connection Server, View Security Server, View Client and the View Agent.
None of the logs I've read gave me any significant errors that would have resolved this for me. Admitted, the logs being in 'full'-trace state, there's a bunch that wasn't exactly clear for me.

I've simplified our environment to ease out the debugging to be as follows:

View Connection Server,
running on Windows Server 2008 R2 (Datacenter) 64bit, VMware View Connection Server 4.6.0-366101,
The checkboxes for both "HTTPS Secure Tunnel : Use Secure Tunnel connection to desktop" and "PCoIP Secure Gateway : Use PCoIP Secure Gateway for PCoIP connections to desktop" have been checked.

View Secure Server,
running on Windows Server 2008 R2 (Datacenter) 64bit, VMware View Security Server 4.6.0-366101,
has been successfully paired with the connection server, and both the "HTTPS Secure Tunnel : External URL" aswell as the "PCoIP Secure Gateway: PCoIP External URL" has been set to an Virtual IP provided in the dmz external firewall with a dst-nat set to the actual ip of the Security Server.

The View Clients are pointed towards the Virtual IP of the View Secure Server.

Since it's not a production environment, I've installed a bunch of Wireshark's to see the traffic;
I've ran traffic snooping on the View Connection Server, View Security Server, View Client and the connected Virtual Desktop
simultaneously and have verified that the PCoIP TCP traffic on port 4172 get's spoken between my client host<->security server and the
securityserver<->virtual desktop just nicely. So the TCP-traffic seems to be tunneled. But what bothers me, is that the wireshark on the Virtual Desktop reveals that the virtual desktop is trying
to talk _all_ the port 4172 UDP-traffic back straight to my View Client host IP. Since this is not allowed by the firewall, the virtual desktop propably fails to work...

But all the scenarios describe that the security server could handle all pcoip-traffic with the view agent (as shown in the Architecture documentation in Figure 5-6), so that no direct connections between the View Client and the View Agent are needed... I just can't get it to work. But it is possible, right?

Any ideas what I could be doing wrong?

Help truly appreciated.

[six4rm]

Are you able to test PCoIP via a non-tunnelled Connection Server?

I've just this week been setting up remote access and ran into an issue with PCoIP. I could connect via RDP remotely no problem, but not PCoIP. All my firewall rules were correct. I then tried to connect via PCoIP across my internal network, which uses a direct connection to the VM (non-tunnelled) and this also failed. I read a few things about having to reinstall the View Agent if VMware Tools has been reinstalled/updated. I'd recently updated to ESX 4.1 U1 and did a recompose of my View VMs having updated VMware Tools, but I hadn't reinstalled the View Agent. After reinstalling the View Agent and recomposing I was then able to connect via PCoIP both internally and remotely.

Just a thought.

[npeter]

Hi,

If you are sure you have configured everything perfect asper to documentation, then posting logs from view client, agent and security server might be of helpful. make sure to include pcoip logs from client,

-noble

[heikkitoivanen]

six4rm:

I had the whole environment inside a same network segment, as a LAN installation, when I first started out with these components.

It worked with PCoIP really nicely, propably because every host could take to eachother without limits; But that was just an experiment on how the user experience would be alltogether. For production, we need to have the secure server tunneling, and the the hosts/components were relocated to different networks. Anyhow, from the LAN experiment, I know that the basic setup is atleast correct and all the pieces can work together in some scenario.

about having to update the agents, that is something I have to look into; I am not the main administrator of our ESX-environment, and I am not 100% sure, which patches have been ran and when. I will have to check that. Thanks for the idea.

[markbenson · Accepted solution]

It does work in the way you've described in your original post. There is no requirement for the virtual desktop to send UDP replies to the client. They will be sent back to the security server which will in turn forward them back to the client.

Something must be set up incorrectly.

Check very carefully the UDP traffic in your wireshark traces. From the client you should see TCP 4172 to the VIP of your SS. You should then see UDP 4172 to the same VIP. You should see reply UDP data from SS to client. The destination port for this reply UDP data should be the source port used for the UDP request. The source port for the reply data should be 4172.

Then check your SS wireshark trace. You should see the same client traffic and you should see a similar PCoIP conversation between SS and the virtual desktop. From the virtual desktop the SS looks like a client. PCoIP UDP should be sent to the SS when it is all set up correctly.

Client --TCP 4172-> SS --TCP 4172-> Virtual Desktop

Client --UDP 4172-> SS --UDP 4172-> Virtual Desktop

Client <-UDP 4172-- SS <-UDP 4172-- Virtual Desktop (the 4172 here is src port, the dst UDP port will be the source port of the UDP request packets above)

If you've verified that you can connect to the same virtual desktop with PCoIP then this problem won't be anything to do with the virtual desktop or the View agent etc.

Double check all your View, network, firewall and NAT settings.

Mark.

[gunnarb]

Mark is spot on in his reply.  But if all else fails shoot me an email: gberger@teradici.com.  I'll get you going.

-Gunnar Berger

Teradici

[heikkitoivanen]

markbenson:

First off, sorry that it took me a week to get back to this, I was out of the office.

Anyway, your reply was really helpful in keeping my mind set in the goal and knowing that it is possible.

I redid some wireshark traces and what really threw me offguard was the fact that I kept seeing the virtual desktop trying to talk all the udp4172 back right to the view client ip, and not the security server. It kept me quite convinced that something is wrong with my configuration of vmware view components.

Anyhow, after systematically verifying step-by-step the conversations you outlined, I noticed that the security server is talking udp4172 to the virtual desktop, but none (udp) is seen coming in when looking at the wireshark trace at the virtual desktop. And I guess that reverted the virtual desktop to try to talk to the original client ip.

As soon as I asked the networking team to re-check their firewall rules, we noticed that they had created a separate service rule out of the pcoip between the security server and the virtual desktops, which only included tcp4172, and not udp4172. Once the udp4172 was added, the virtual desktop started getting the packets from the security server, and that got the virtual desktop to send its udp4172 back to the security servers (inside) ip, and not to the client's ip.

So all in all, got this to work, and it's working great!

thank you all for your help!

[markbenson]

Yes. Blocking PCoIP 4172 (UDP or TCP) at your firewall will absolutely have been the cause of your black screen. In fact if any of the 3 setup steps here http://communities.vmware.com/docs/DOC-14974 are not done correctly you will get the black screen (step 3 in your case).

I'm glad you've fixed your firewall configuration and now have it working.

Thanks for posting back. Your experience will surely help others too.

Mark.

[heikkitoivanen]

I'll also link this http://communities.vmware.com/message/1784660#1784660 in reference for the

checklist I made for myself which helped me pinpoint my issue; Maybe it will help some others facing a similar issue.