VMware Horizon UAG "Unable to connect to server: target machine actively refused it."

[cworthen]

We have a lingering issue that has been hard to pinpoint the problem.  Initially, about once a month we would receive a notice in the middle of the night from our monitoring system that the external connections for our VDI environment is down.  When trying to connect to the servers via the Horizon Client we would receive this error message (screenshot attached):

    "Unable to connect to server: No connection could be made because the target machine actively refused it."

If we rebooted the UAGs that were responsible for the connection the issue would go away.  Eventually the issue began occurring once every few days.  We opened a ticket and were told to upgrade our UAGs since we were on an older version (UAG 2.9).  We have since upgraded and the issue went away for a few months but is now occurring again a few times a week.  A simple reboot fixes the problem and it never happens during the day but a daily reboot seems like an overkill solution for this problem.

Environment:

vCenter Appliance 6.7 Update 3n (6.7.0.48000)

ESXi 6.7 P05 (17700523)

Horizon 7.13

    2 Connection/Admin Servers

Unified Access Gateway Appliance v21.03.1

    4 total UAGs (2 in HA Group pointing to Connection Server 1 and 2 in HA Group pointing to Connection Server 2)

Windows 10 2004, 20H2 & 21H1

[SurajRoy]

From the issue description, it seems it can be misconfigured UAG or incorrect health monitoring.

If rebooting the UAG fix the issue, it means  it might have reached 2000 session bench mark and only way to clear the old stale session is to reboot.

NOTE: UAG by default support 2000 session. This 2000 session not only include VDI session but also health monitoring probing from 3rd party device.

you can enable the log level on UAG to Debug and when issue happen, before rebooting the UAG, download the log bundle. as rebooting will clear the events.

Inside the log bundle you will find Active Session file. Open it and count the number of session. Also check the esmanager log for the more specific cause of the issue.