Hi community, this problem has been bugging me for a while now so I figured I'd reach out to the community and hopefully get this thing working! Bear with me, this is a long one....
I work for a service provider and want to building out a scalable UAG-pair-per-tenant design behind a pair of HAProxy load-balancers for external (i.e. over the Internet) access. Note that we are using the Horizon DaaS product not the Horizon View product. Apart from the difference in name, I believe the UAG functions identically in both environments.
My final solution would be two pfSense (community edition) firewalls with the HAProxy package installed on both to provide HA and load-balancing functionality to the tenant UAGs behind them. As we onboard more tenants, I would add another pair of UAGs for each. HAProxy will selectively pick which UAG is required for the incoming connection based on SNI. For example:
Connections arriving at daas.tenanta.com will be directed to a UAG pair for tenantA. Connections arriving at daas.tenantb.com will be directed to a UAG pair for tenantB. And so on and so forth.
As this is my first attempt at building out a HAProxy solution I've decided to keep it simple and use just one pfSense/HAproxy load-balancer, one UAG and one tenant. This is what I have now:
In this given configuration, I've observed the following:
Some things I'm not sure about:
Some things I've tried:
I'm really at my wits end here so any help would be much appreciated!
I don't know specifically which protocol you are using because you list all 3 in your rule, but in the case of PCoIP, HAproxy does not support UDP. You will have to use the 3 external IP setup. One for the HAProxy and one for each UAG per tenant.
Take a look at this thread: HAproxy loadbalancer and View 6.1.1
Hope that helps you get past or at least answer your question.
Larry
Hi Larry,
Thanks for your response. I did read that thread prior to posting - it contains a wealth of knowledge. I guess it's good to know that it is possible but using method 2 where the LB is used for the initial connection then all subsequent traffic goes direct to the UAGs.
If that is really the case then that's a shame with HAProxy. It's interesting that the LB product developed by loadbalancer.org seems to support UDP even though it's purportedly built on HAProxy (http://pdfs.loadbalancer.org/Vmware_Horizon_Deployment_Guide.pdf ) Perhaps they've incorporated bits of IPVS which does layer 4 load-balancing into their product.
It also seems a little strange that I can't build a connection using the Horizon Client after authenticating. i.e. I don't get a list of entitled desktops. I wouldn't have thought this would be using UDP.
As for which protocols I am using, I was hoping to use both Blast and PCoIP. This means a largely seamless experience for both internal and external users with no need to dictate which protocol gets used depending on where they're connecting from. They just fire up the Horizon View client and connect (obviously assuming the FQDN is the same both internally and externally).
I'll try method 2 and let you know how I get on.
Thanks,
Kam
The browser works because you are essentially reverse proxying through the UAG to the connection servers web portal. When you establish a connection to the backend resource (desktop/RDSH) through the portal you are connecting from the connection server.
The error you mention has me suspecting that you may not have the appropriate ports open from the UAG to the resources (desktops / RDSH) on the backend. If you review the logs on the UAG should indicate the problem.
Since you can authenticate using the client and connect successfully to the HTML portal it sounds like your HAProxy is setup correctly.
Thanks for your reply. I don't believe it's a connection problem between the UAG and the desktop resources as they're all on the same network - i.e. for now, there is no inner-firewall.
After reading Network Ports in VMware Horizon 7: VMware Horizon 7 version 7.2 it's clear why the browser works - all communication happens over TCP - there is no UDP. And as was already mentioned, HAProxy, for better or for worse, does not do UDP. As mentioned specifically in the HAProxy version 1.7.11 - Starter Guide, it says HAProxy "will not see IP packets nor UDP datagrams". Major bummer
I do find it interesting that when I do a packet capture of a NAPT, non-loadbalancing connection, I do not see any UDP packets (PCoIP 4172) until the actual display starts up. This corresponds to what is mentioned in the Network Ports article in the link above - that all login traffic happens over TCP 443. So if there are no UDP packets during the login stage, then why can I not even bring up the list of available desktops when using the Horizon View client and through a load-balanced connection? Perhaps markbenson can help as he authored this Load Balancing across VMware Unified Access Gateway Appliances and helped immensely in Can BEAT run over a different port than UDP 8443?
Assuming I somehow resolve the "could not establish tunnel" issue, I could have two connection server entries - one for external (uses Blast only) and one for internal which has no protocol restrictions.
I guess for now in order to retain the consistent "one connection server" user experience, it looks like I will need to go down the multiple VIP method instead.
KFM - you certainly need to fix your Horizon tunnel issue first. HTML Access (browser) doesn't use the tunnel, but native Horizon clients do. After authentication, Horizon clients establish the tunnel connection using the value of tunnelExternalUrl. e.g. https://daas.tenanta.com:443. This is one of the secondary connections and you MUST ensure that gets routed to the same UAG as used for the initial primary connection. If the tunnelExternalUrl is not usable by the client, the tunnel connection is blocked or misrouted to the wrong UAG appliance it will fail. You then won't get the list of entitled desktops. All secondary protocols (Tunnel, Blast, PCoIP) must be routed to the same UAG as the primary connection.
Hello KFM,
Do you have a sollution for you HAProxy issues ? We have 2 pod and one UAG and no LB installed jet, but sometimes we have dubble logon issues, We think it because of round robin without LB. So we are looking for a good/free load balancer that does the job.
So yes, I did get it working with HAProxy, albeit only one as I didn't get time to deploy pfSense/HAProxy in a HA setup. I also did get it to load-balance between two UAGs. I can send you the HAProxy configuration if you need it for reference? I never got any double login issues.
In a nutshell, HAProxy will not work with UDP which means that we cannot use Method 1 - Source IP Affinity. I did get it working with Method 2 - Multiple Port Number Groups and also Method 3 - Multiple VIPs. The configuration for those three methods are here: Load Balancing across VMware Unified Access Gateway Appliances.
Hello,
If you can send me the config i will appreciate that.
I saw that you put the load balancer in front of the UAG. I was thinking to put the load balancer between UAG and the connection server.
Bert
The intention was to have two UAGs in an active/active setup. This requires a load-balancer in front to distribute the incoming sessions to the UAG with the least connections.
As I was deploying the UAG for a Horizon DaaS deployment, I didn't need to put another LB in front of the tenant appliances since these are natively HA out of the box - no further configuration or LB required. I haven't worked with Horizon View for many years so I can't say if putting a LB in front of the connection server is a supported topology.
As for the HAProxy config, here it is below. Note that the design I came up with was to have one public IP address upon which all tenant portal URLs would resolve to. HAProxy would then use SNI to forward the request to that particular tenant's UAG pair. This allowed me to scale out the number of tenant appliances whilst LB the connections through a pair of pfSense/HAProxy appliances. There are obviously a number of different ways you could design this - each with their respective pros and cons.
Hope that helps!
# Automaticaly generated, dont edit manually.
# Generated on: 2019-04-19 08:26
global
maxconn 1000
log /var/run/log local0 info
stats socket /tmp/haproxy.socket level admin
uid 80
gid 80
nbproc 1
hard-stop-after 15m
chroot /tmp/haproxy_chroot
daemon
server-state-file /tmp/haproxy_server_state
listen HAProxyLocalStats
bind 127.0.0.1:2200 name localstats
mode http
stats enable
stats refresh 2
stats admin if TRUE
stats show-legends
stats uri /haproxy/haproxy_stats.php?haproxystats=1
timeout client 5000
timeout connect 5000
timeout server 5000
frontend Universal_UAG_Frontend
bind publicIP:443 name publicIP:443
mode tcp
log global
option tcplog
timeout client 30000
tcp-request inspect-delay 5s
acl TenantA req.ssl_sni -i daas.TenantA.com
acl TenantB req.ssl_sni -i daas.TenantB.com
tcp-request content accept if { req.ssl_hello_type 1 }
use_backend TenantA_UAG_Pool_ipvANY if TenantA
use_backend TenantB_UAG_Pool_ipvANY if TenantB
backend TenantA_UAG_Pool_ipvANY
mode tcp
id 100
log global
stick-table type ip size 50k expire 1h
stick on src
balance leastconn
timeout connect 30000
timeout server 30000
retries 3
option httpchk GET /favicon.ico
server TenantA-UAG1 TenantA_UAG1_PrivateIP:443 id 106 check-ssl check inter 1000 verify none
server TenantA-UAG2 TenantA_UAG2_PrivateIP:443 id 102 check-ssl check inter 1000 verify none
backend TenantB_UAG_Pool_ipvANY
mode tcp
id 103
log global
stick-table type ip size 50k expire 1h
stick on src
balance leastconn
timeout connect 30000
timeout server 30000
retries 3
option httpchk GET /favicon.ico
server TenantB-UAG1 TenantB_UAG1_PrivateIP:443 id 101 check-ssl check inter 1000 verify none
server TenantB-UAG2 TenantB_UAG2_PrivateIP:443 id 102 check-ssl check inter 1000 verify none
Thank you for sharing, i will take a look at it.
hi,
Five things to consider when you deploy loadbalancing in front of UAG.
1- make usre the secure tunnel option is checked in the UAG settings.
2- In Layer 7 mode (full ssl proxy), the certificate deployed on UAG MUST be the same as the one deployed on the haproxy.
3- In the UAG, you have to indicate the public IP address used by the clients.
4- You have to consider 3 ports (4 if you want to deploy blast). tcp/443, tcp/4172 and udp/4172. this latter is for pcoip.
5- Use the session persistency based on source IP, so the client connection is stuck to the same backend UAG.
keep in mind that all communications between external clients and connection servers or virtual desktops are ALL going through UAG. therefore, you have to make sure that your pfsense haproxy has the required performance to handle all the traffic.
the F5 deployment may provide more insight.... https://www.f5.com/pdf/deployment-guides/vmware-horizon-view-dg.pdf
cheers.
heloma.
I know this is an old thread but wanted to point out, the Horizon Client does not provide SNI information. So when your ACL is looking for an SNI, and you have no default backend definied - it's going to ultimately fail.
That very first connection from the client does - in order to vet the certificate. When you go to use a resource however, it does not. Ultimately the "trick" is just to have the default backed blindly point to your pool of UAGs.
You can verify by simply disabling the UDP tunnel and configuring UAG to force everything through TCP/443 - client still won't work properly. I had an article save somewhere, but vmware's response was "tough cookies".